In the fields of nonlinear optics and fluid dynamics, modulational instability or sideband instability is a phenomenon whereby deviations from a periodic waveform are reinforced by nonlinearity, leading to the generation of spectral-sidebands and the eventual breakup of the waveform into a train of pulses.[1][2][3]

It is widely believed that the phenomenon was first discovered − and modeled − for periodic surface gravity waves (Stokes waves) on deep water by T. Brooke Benjamin and Jim E. Feir, in 1967.[4] Therefore, it is also known as the Benjamin−Feir instability. However, spatial modulation instability of high-power lasers in organic solvents was observed by Russian scientists N. F. Piliptetskii and A. R. Rustamov in 1965,[5] and the mathematical derivation of modulation instability was published by V. I. Bespalov and V. I. Talanov in 1966.[6] Modulation instability is a possible mechanism for the generation of rogue waves.[7][8]

## Initial instability and gain

Modulation instability only happens under certain circumstances. The most important condition is anomalous group velocity dispersion, whereby pulses with shorter wavelengths travel with higher group velocity than pulses with longer wavelength.[3] (This condition assumes a focusing Kerr nonlinearity, whereby refractive index increases with optical intensity.)[3]

The instability is strongly dependent on the frequency of the perturbation. At certain frequencies, a perturbation will have little effect, whilst at other frequencies, a perturbation will grow exponentially. The overall gain spectrum can be derived analytically, as is shown below. Random perturbations will generally contain a broad range of frequency components, and so will cause the generation of spectral sidebands which reflect the underlying gain spectrum.

The tendency of a perturbing signal to grow makes modulation instability a form of amplification. By tuning an input signal to a peak of the gain spectrum, it is possible to create an optical amplifier.

### Mathematical derivation of gain spectrum

The gain spectrum can be derived [3] by starting with a model of modulation instability based upon the nonlinear Schrödinger equation[clarification needed]

${\displaystyle {\frac {\partial A}{\partial z))+i\beta _{2}{\frac {\partial ^{2}A}{\partial t^{2))}=i\gamma |A|^{2}A,}$

which describes the evolution of a complex-valued slowly varying envelope ${\displaystyle A}$ with time ${\displaystyle t}$ and distance of propagation ${\displaystyle z}$. The imaginary unit ${\displaystyle i}$ satisfies ${\displaystyle i^{2}=-1.}$ The model includes group velocity dispersion described by the parameter ${\displaystyle \beta _{2))$, and Kerr nonlinearity with magnitude ${\displaystyle \gamma .}$ A periodic waveform of constant power ${\displaystyle P}$ is assumed. This is given by the solution

${\displaystyle A={\sqrt {P))e^{i\gamma Pz},}$

where the oscillatory ${\displaystyle e^{i\gamma Pz))$ phase factor accounts for the difference between the linear refractive index, and the modified refractive index, as raised by the Kerr effect. The beginning of instability can be investigated by perturbing this solution as

${\displaystyle A=\left({\sqrt {P))+\varepsilon (t,z)\right)e^{i\gamma Pz},}$

where ${\displaystyle \varepsilon (t,z)}$ is the perturbation term (which, for mathematical convenience, has been multiplied by the same phase factor as ${\displaystyle A}$). Substituting this back into the nonlinear Schrödinger equation gives a perturbation equation of the form

${\displaystyle {\frac {\partial \varepsilon }{\partial z))+i\beta _{2}{\frac {\partial ^{2}\varepsilon }{\partial t^{2))}=i\gamma P\left(\varepsilon +\varepsilon ^{*}\right),}$

where the perturbation has been assumed to be small, such that ${\displaystyle |\varepsilon |^{2}\ll P.}$ The complex conjugate of ${\displaystyle \varepsilon }$ is denoted as ${\displaystyle \varepsilon ^{*}.}$ Instability can now be discovered by searching for solutions of the perturbation equation which grow exponentially. This can be done using a trial function of the general form

${\displaystyle \varepsilon =c_{1}e^{ik_{m}z-i\omega _{m}t}+c_{2}e^{-ik_{m}^{*}z+i\omega _{m}t},}$

where ${\displaystyle k_{m))$ and ${\displaystyle \omega _{m))$ are the wavenumber and (real-valued) angular frequency of a perturbation, and ${\displaystyle c_{1))$ and ${\displaystyle c_{2))$ are constants. The nonlinear Schrödinger equation is constructed by removing the carrier wave of the light being modelled, and so the frequency of the light being perturbed is formally zero. Therefore, ${\displaystyle \omega _{m))$ and ${\displaystyle k_{m))$ don't represent absolute frequencies and wavenumbers, but the difference between these and those of the initial beam of light. It can be shown that the trial function is valid, provided ${\displaystyle c_{2}=c_{1}^{*))$ and subject to the condition

${\displaystyle k_{m}=\pm {\sqrt {\beta _{2}^{2}\omega _{m}^{4}+2\gamma P\beta _{2}\omega _{m}^{2))}.}$

This dispersion relation is vitally dependent on the sign of the term within the square root, as if positive, the wavenumber will be real, corresponding to mere oscillations around the unperturbed solution, whilst if negative, the wavenumber will become imaginary, corresponding to exponential growth and thus instability. Therefore, instability will occur when

${\displaystyle \beta _{2}^{2}\omega _{m}^{2}+2\gamma P\beta _{2}<0,}$   that is for   ${\displaystyle \omega _{m}^{2}<-2{\frac {\gamma P}{\beta _{2))}.}$

This condition describes the requirement for anomalous dispersion (such that ${\displaystyle \gamma \beta _{2))$ is negative). The gain spectrum can be described by defining a gain parameter as ${\displaystyle g\equiv 2|\Im \{k_{m}\}|,}$ so that the power of a perturbing signal grows with distance as ${\displaystyle P\,e^{gz}.}$ The gain is therefore given by

${\displaystyle g={\begin{cases}2{\sqrt {-\beta _{2}^{2}\omega _{m}^{4}-2\gamma P\beta _{2}\omega _{m}^{2))},&{\text{for ))\displaystyle \omega _{m}^{2}<-2{\frac {\gamma P}{\beta _{2))},\\[2ex]0,&{\text{for ))\displaystyle \omega _{m}^{2}\geq -2{\frac {\gamma P}{\beta _{2))},\end{cases))}$

where as noted above, ${\displaystyle \omega _{m))$ is the difference between the frequency of the perturbation and the frequency of the initial light. The growth rate is maximum for ${\displaystyle \omega ^{2}=-\gamma P/\beta _{2}.}$

## Modulation instability in soft systems

Modulation instability of optical fields has been observed in photo-chemical systems, namely, photopolymerizable medium.[9][10][11][12] Modulation instability occurs owing to inherent optical nonlinearity of the systems due to photoreaction-induced changes in the refractive index.[13] Modulation instability of spatially and temporally incoherent light is possible owing to the non-instantaneous response of photoreactive systems, which consequently responds to the time-average intensity of light, in which the femto-second fluctuations cancel out.[14]

## References

1. ^ Benjamin, T. Brooke; Feir, J.E. (1967). "The disintegration of wave trains on deep water. Part 1. Theory". Journal of Fluid Mechanics. 27 (3): 417–430. Bibcode:1967JFM....27..417B. doi:10.1017/S002211206700045X. S2CID 121996479.
2. ^ Benjamin, T.B. (1967). "Instability of Periodic Wavetrains in Nonlinear Dispersive Systems". Proceedings of the Royal Society of London. A. Mathematical and Physical Sciences. 299 (1456): 59–76. Bibcode:1967RSPSA.299...59B. doi:10.1098/rspa.1967.0123. S2CID 121661209. Concluded with a discussion by Klaus Hasselmann.
3. ^ a b c d Agrawal, Govind P. (1995). Nonlinear fiber optics (2nd ed.). San Diego (California): Academic Press. ISBN 978-0-12-045142-5.
4. ^ Yuen, H.C.; Lake, B.M. (1980). "Instabilities of waves on deep water". Annual Review of Fluid Mechanics. 12: 303–334. Bibcode:1980AnRFM..12..303Y. doi:10.1146/annurev.fl.12.010180.001511.
5. ^ Piliptetskii, N. F.; Rustamov, A. R. (31 May 1965). "Observation of Self-focusing of Light in Liquids". JETP Letters. 2 (2): 55–56.
6. ^ Bespalov, V. I.; Talanov, V. I. (15 June 1966). "Filamentary Structure of Light Beams in Nonlinear Liquids". ZhETF Pisma Redaktsiiu. 3 (11): 471–476. Bibcode:1966ZhPmR...3..471B. Archived from the original on 31 July 2020. Retrieved 17 February 2021.
7. ^ Janssen, Peter A.E.M. (2003). "Nonlinear four-wave interactions and freak waves". Journal of Physical Oceanography. 33 (4): 863–884. Bibcode:2003JPO....33..863J. doi:10.1175/1520-0485(2003)33<863:NFIAFW>2.0.CO;2.
8. ^ Dysthe, Kristian; Krogstad, Harald E.; Müller, Peter (2008). "Oceanic rogue waves". Annual Review of Fluid Mechanics. 40 (1): 287–310. Bibcode:2008AnRFM..40..287D. doi:10.1146/annurev.fluid.40.111406.102203.
9. ^ Burgess, Ian B.; Shimmell, Whitney E.; Saravanamuttu, Kalaichelvi (2007-04-01). "Spontaneous Pattern Formation Due to Modulation Instability of Incoherent White Light in a Photopolymerizable Medium". Journal of the American Chemical Society. 129 (15): 4738–4746. doi:10.1021/ja068967b. ISSN 0002-7863. PMID 17378567.
10. ^ Basker, Dinesh K.; Brook, Michael A.; Saravanamuttu, Kalaichelvi (2015). "Spontaneous Emergence of Nonlinear Light Waves and Self-Inscribed Waveguide Microstructure during the Cationic Polymerization of Epoxides". The Journal of Physical Chemistry C. 119 (35): 20606–20617. doi:10.1021/acs.jpcc.5b07117.
11. ^ Biria, Saeid; Malley, Philip P. A.; Kahan, Tara F.; Hosein, Ian D. (2016-03-03). "Tunable Nonlinear Optical Pattern Formation and Microstructure in Cross-Linking Acrylate Systems during Free-Radical Polymerization". The Journal of Physical Chemistry C. 120 (8): 4517–4528. doi:10.1021/acs.jpcc.5b11377. ISSN 1932-7447.
12. ^ Biria, Saeid; Malley, Phillip P. A.; Kahan, Tara F.; Hosein, Ian D. (2016-11-15). "Optical Autocatalysis Establishes Novel Spatial Dynamics in Phase Separation of Polymer Blends during Photocuring". ACS Macro Letters. 5 (11): 1237–1241. doi:10.1021/acsmacrolett.6b00659. PMID 35614732.
13. ^ Kewitsch, Anthony S.; Yariv, Amnon (1996-01-01). "Self-focusing and self-trapping of optical beams upon photopolymerization" (PDF). Optics Letters. 21 (1): 24–6. Bibcode:1996OptL...21...24K. doi:10.1364/ol.21.000024. ISSN 1539-4794. PMID 19865292.
14. ^