The future of mobile optics, Apple patent filings

Published by Sergey Sidorovich.

In recent years a key differentiator in the competition between smart phone manufacturers has been cameras. A quick look at the marketing for the Apple iphone, Samsung Galaxy, LG, Nokia and  HTC  smart phones demonstrates how important outstanding camera performance has become to phone vendors. Consumers  expect high quality, high resolution cameras for both the front faces and rear facing sensors. As a result camera module suppliers are constantly trying to improve maximum achievable resolution, sensitivity, color reproduction,  illumination, hand shake correction and auto focusing.

One industry with a strong interest in this trend is suppliers who add custom optics on top of a cell phone for specific applications. These applications include consumer photography (Photojojo), medical devices like otoscopes (CellScope) and  inspect eye health (SmartVision) These add-on providers need to keep a sharp eye on changes in camera optics and mechanics  as they could have a  big impact on their own product road maps. 

One of the key technical obstacles that must be overcome to improve image quality is the mechanical space limitations. The thickness of a mobile phone very seldom exceeds 10 mm. And the sizes of mobile optics used inside of cameras are correspondingly small. Design of optics which could fit into this size is not always possible. In fact, the laws of physics are so inconvenient that Apple, which is known for its love to sleek industrial design, downplayed the fact that even on the iphone a 'bump out' was needed to provide the proper space for camera optics.

So a  recent Apple patent that suggests several ideas to reduce camera size caught our eye.

Solutions offered by Apple

Tilted Mirror 

Mirror is placed in front part of the Lens at an angle 45 degrees relative to optical axis. Such placement of the mirror enables to change direction of the optical axis on 90 degrees. As result, lenses can be placed along the mobile device case. This means that length of the lens is not limited by thickness of the case. According to the patent, thickness of the camera can reach 32 mm with suggested mirror. Longer optical system makes it possible to solve several problems of optical quality improvement and expansion of the devices functionality.

Image was taken from Apple patent (US 8,988,564 B2) and edited by OFH for clarity.

An obvious benefit in this approach is that it makes it possible to use greater number of optical elements. This change allows for more elements with simpler shapes to be used and for better optical performance. 

Greater length of the lens assumes that the angles of propagation of rays relative to the optical axis become smaller. This reduces the amount of glares and their intensity. Probability of ghost images reduces. Loss of light caused by scattering on the lenses should go down too.

Also greater length makes it possible to design structure for providing of the optical zoom. The patent describes a method using movable lenses. With help of this focus of the lens can be changed in a fairly wide range. 

Image was taken from Apple patent (US 2015/0042870 A1) and edited by OFH for clarity.

Another potential advantage of the mirror is possibility of its using for image stabilization. Method which enables to realize this idea had been expounded in detail in other patent. Magnet and electrical coils are used for control of the mirror tilt. Feedback carried out by means of the Hall sensors. It is assumed that the proposed structure will significantly reduce the effect of camera shake on quality of the resulting picture.

Beam-splitting prism

The patent also described a beam-splitting prism placed in back of the camera optical system. It consists of four triangular prisms. The basis of each prism is an isosceles right triangle. The main purpose of this prism is in separation the optical channel into three parts.

Image was taken from Apple patent (US 8,988,564 B2) and edited by OFH for clarity.

Each of the three channels is designed to work with certain range of wavelength band (color) - red, blue and green. For each channel is provided separate sensor. Color images which are received with help of such methods are result of processing of the three separate images.

Such approach to the image acquisition and its processing make it possible to relinquish Bayer filters. This leads to increasing of the light amount on each pixel almost three times. As result camera sensitivity is significantly increased. Moreover many other problems associated with the Bayer filter, are solved with help of light separation method. This is one more reason why growth of camera resolution can be expected.

The developers noted that the prism could be used not only for color separation. Wavelength band can be shifted (widened) in IR (UV) light region. Beam-splitting prism can be modernized to separate IR (UV) light from visible light. Another mentioned in patent method of application is polarization imaging without light losing.

Potential drawbacks

Should be noted that presented optical scheme has some potential drawbacks. Here are a few:

Thickness of the mobile device determines the maximum diameter of the lens

In case when camera lens is situated along of the mobile device case, maximal diameter of its optical elements is limited by thickness of the device case. It is likely the field of view of the cameras installed in the thin mobile device will have even less field of view in current layouts. Uniformity of illumination in image plane also could be a problem because of small lens diameter.

Thickness of the mobile device determines the maximum size of the sensor

Separation of the light flux on three parts presupposes using of three separate sensors. At least one of them will be placed perpendicularly to the mobile device. This means that size of this sensor should not exceed thickness of the mobile device, which could be problematic especially for very thin gadgets.

Mobility of the entrance pupil

Entrance pupil of the camera optical system moves during zoom process according the patent. For producers of optical attachments for mobile devices this will be frustrating. For best attachment performance, the exit pupil of the attachment should coincide with the entrance pupil of the camera lens. Thus, if the entrance pupil of the camera is moved, then the exit pupil of the additional device should move with it. Otherwise, the appearance of the high degree irregularity of illumination in image plane and even cropping of the image is likely.

The way out of this difficulty may consist in fixing the focus of the mobile camera. As result entrance pupil remains motionless and additional device can work without problem. Or optical engineers could design adaptive construct with feedback which could watch over entrance pupil of the mobile camera and change position of the exit pupil of the optical attachment. Of course, this will make design of such devices more complex and expensive.


The design and the ideas presented in the Apple patent are an attempt to deal with some of the fundamental limitations of small optics . Despite the fact that the proposed design shows the way out from a number of limitations inherent in conventional cameras, they have some drawbacks. It is possible that the bright minds working at Apple, will be able to neutralize all the shortcomings and new digital cameras will satisfy the most demanding users