Post-Processing – In-Depth Camera Distortion Effects
Post-processing is a technology that performs algorithmic processing on the original game screen to improve the quality of the screen or enhance the effect of the screen, which can be achieved through the shader Shader program.
Overview
Deformation special effect is a type of post-processing technology that processes and enhances image effects. It is often used in the special effects of various camera short video apps, such as beauty and slimming, and funhouse mirror effects.
Deformation effect of beauty camera
This article mainly sorts out the following commonly used deformation effects from various beauty cameras:
- Local distortion (twirl effect)
- Local inflation (inflate effect)
- Squeeze in any direction (pinch effect)
Among them, twisting can be used for local rotation of the eyes, expansion can be used for big eyes, extrusion/stretching can be used for face plasticization and face thinning, etc. How to achieve these deformations through the shader Shader is the focus of this article. (The source code is at the end of the article.)
The Principle of Deformation Technology
Although the effects of deformation are all kinds of strange, they are often inseparable from these three elements: deformation position, influence range and deformation degree.
Deformation Shader realizes character dance
Therefore, its implementation in Shader is to construct a deformation function, which will pass in the original UV coordinates, the deformed position, Range, and Strength, and then generate the deformed sampling coordinates after calculation. The code is as follows:
#iChannel0 "src/assets/texture/joker.png"
vec2 deform(vec2 uv, vec2 center, float range, float strength) {
// TODO: deformation process
return uv;
}
void mainImage(out vec4 fragColor, vec2 coord) {
vec2 uv = coord / iResolution.xy;
vec2 mouse = iMouse.xy / iResolution.xy;
uv = deform(uv, mouse, .5, .5);
vec3 color = texture(iChannel0, uv).rgb;
fragColor.rgb = color;
}
The shader code in this article adopts the GLSL specification and follows the Shader-Toy writing method, which is convenient for everyone to preview.
Deformation Tips: Sampling Distance Field Transform
We set the fixed-point coordinate O, the distance from any point to the point O is dist, with different dist values as the radius, with the point O as the center, an infinite number of equidistant sampling circles can be formed, which are called the distance field of the point O.
Sampling distance field
We can change the texture sampling position by changing the size or position of the sampling circle to achieve the deformation effect of expansion/contraction and extrusion.
vec2 deform(vec2 uv, vec2 center, float range, float strength) {
float dist = distance(uv, center);
vec2 direction = normalize(uv - center);
dist = transform(dist, range, strength); // Change the sampling circle radius
center = transform(center, dist, range, strength); // Change the center position of sampling circle
return center + dist * direction;
}
More details in the application of this technique will not be discussed here, and now we will start with a simple distortion.
Distortion
The distortion effect is similar to the vortex shape, and the characteristic the closer the center point is, the more intense the rotation is. We can express the relationship between the distance d from the center point and the corresponding rotation angle θ through a decreasing function.
As shown in the figure below, a simple linear function θ=-A/R*d+A is used, where A represents the rotation angle of the twisted center, A is a positive number, it means the rotation direction is clockwise, a negative number means counterclockwise, and R means the boundary of the twist;
Distortion Principle
As shown in the figure above, the input parameters of the twist function A (center rotation angle Angle) and R (deformation range Range) can be described as follows:
1) A represents the center rotation angle, the larger the absolute value, the higher the degree of distortion;
2) A>0 means the distortion direction is clockwise, otherwise A<0 means counterclockwise;
3) R represents the twisted boundary, the larger the value, the larger the influence range.
Distortion Dynamics Effects
We can introduce the time variable Time to dynamically change the value of A to generate a twisting effect, such as the clown twisting effect in the figure above. The specific Shader code is as follows:
#iChannel0 "src/assets/texture/joker.png"
#define Range .3
#define Angle .5
#define SPEED 3.
mat2 rotate(float a) // rotation matrix
{
float s = sin(a);
float c = cos(a);
return mat2(c,-s,s,c);
}
vec2 twirl(vec2 uv, vec2 center, float range, float angle) {
float d = distance(uv, center);
uv -=center;
// d = clamp(-angle/range * d + angle,0.,angle); // linear equation
d = smoothstep(0., range, range-d) * angle;
uv *= rotate(d);
uv+=center;
return uv;
}
void mainImage(out vec4 fragColor, vec2 coord) {
vec2 uv = coord / iResolution.xy;
vec2 mouse = iMouse.xy / iResolution.xy;
float cTime = sin(iTime * SPEED);
uv = twirl(uv, mouse, Range, Angle * cTime);
vec4 color = texture(iChannel0, uv);
fragColor = color;
}
It is worth mentioning that, in addition to expressing the distortion relationship with a linear equation, the smoothstep method can also be used. Compared with the linear function, the smoothstep method is smoother at the distortion boundary, as shown in the following figure:
Comparison of linear and smoothstep distortion equation effects
Considering the smoothness of the boundary, the following deformation methods will also use the smoothstep function instead of the linear equation.
Expansion/Contraction
Dilation Features Textures close to the dilation center are stretched, and textures close to the dilation boundary are squeezed, which means that within the dilation range, with the dilation center as the distance field, each sampled circle should have a smaller radius than the original, and the circle spacing gradually expands from inside to outside.
As shown on the right side of the figure below, we make the spacing between the new sampling circles monotonically increase from the inside to the outside by mapping the equally spaced black sampling circles to the more cohesive red sampling circles.
Dilated Sampling Distance Field Transform
We sample the smoothly increasing function smoothstep to calculate the scale value scale by sampling the circle radius dist:
The function in the above figure shows that near the center of expansion, the sampling circle has the most obvious scaling, and the scaling value is the smallest (1-S). 1. The sampling circle is no longer scaled.
float scale = (1.- S) + S * smoothstep(0.,1., dist / R); // Calculate the expansion sampling radius scaling value
So we get the above sampling radius scaling formula, where Strength (0<S<1) is set to represent the degree of expansion. For the transformation process of the expansion distance field, it is easy to infer that to achieve the reverse effect of expansion and contraction, it is enough to directly let S be in the [-1,0] interval.
The S value corresponds to the degree of expansion and contraction Strength
As shown in the figure above, the parameters S (Deformation Strength) and R (Deformation Range) of the expansion function can be described as follows:
1) When S is in the [0,1] interval, the expansion effect is exhibited. The larger the value of S, the greater the degree of expansion.
2) When S is in the range of [-10], it shows a shrinkage effect, the smaller the value of S, the higher the degree of shrinkage;
3) R represents the boundary of deformation, the larger the value, the larger the affected area;
Dynamic Expansion Effect
We can introduce the time variable Time to dynamically change the value of Strength to simulate the breathing animation, as shown in the figure above, the clown bulging effect. The specific Shader code is as follows:
#iChannel0 "src/assets/texture/joker.png"
#define SPEED 2. // Speed
#define RANGE .2 // Range
#define Strength .5 * sin(iTime * SPEED) // Strength
vec2 inflate(vec2 uv, vec2 center, float range, float strength) {
float dist = distance(uv , center);
vec2 dir = normalize(uv - center);
float scale = 1.-strength + strength * smoothstep(0., 1. ,dist / range);
float newDist = dist * scale;
return center + newDist * dir;
}
void mainImage(out vec4 fragColor, vec2 coord) {
vec2 uv = coord / iResolution.xy;
vec2 mouse = iMouse.xy / iResolution.xy;
uv = inflate(uv, mouse, RANGE, Strength);
vec3 color = texture(iChannel0, uv).rgb;
fragColor.rgb = color;
}Longitudinal/Horizontal Stretch
Original Image – Longitudinal Stretching – Horizontal Stretching – Expansion
The previous expansion is achieved by scaling the distance field sampling circle, and the vertical/horizontal stretching is to only scale the x-axis or y-axis of the sampling circle, which is generally used for the “long leg effect” of beauty.
Longitudinal Stretching Distance Field Transform
It can be found that the laterally stretched distance field is transformed into multiple elliptical sampling circles, and the code is implemented as follows:
vec2 inflateX(vec2 uv, vec2 center, float radius, float strength) {
// The previous code is the same as the inflation implementation
...
return center + vec2(newDist, dist) * dir; // horizontal stretching scale only applies to the x axis
}Extrusion
Extrusion generally specifies an action point and an extrusion direction, which is characterized by pushing the texture near the action point to the extrusion end position.
As shown in the figure below, the green action point P is the starting point of extrusion, the arrow is the extrusion vector V, where the vector direction indicates the extrusion direction, the vector length (V) represents the extrusion distance, and the vector endpoint is the position after extrusion.
To achieve texture extrusion, the center of the sampling circle is offset to the extrusion vector V, and the sampling center point should be translated to the position of point P.
Extrusion uses distance field transform
As the radius dist of the sampling circle gradually increases from the inside to the outside, the transformed center offset gradually shortens, we can use the -smoothstep smooth decreasing function to process the relationship between the sampling circle radius dist and the circle offset.
Formula: offset=length(V)-length(V)*smoothstep(0,R,dist), where R represents the extrusion boundary Range.
Extrusion Dynamic Effect
Similarly, we introduce the time variable Time to dynamically change the length and direction of the extrusion vector, which can realize the jitter effect, such as the clown’s crotch effect as shown above. The specific Shader code is as follows:
#iChannel0 "src/assets/texture/joker.png"
#define RANGE .25 // Range
#define PINCH_VECTOR vec2( sin(iTime * 10.), cos(iTime * 20.)) * .03 // Pinch Vector
vec2 pinch(vec2 uv, vec2 targetPoint, vec2 vector, float range)
{
vec2 center = targetPoint + vector;
float dist = distance(uv, targetPoint);
vec2 point = targetPoint + smoothstep(0., 1., dist / range) * vector;
return uv - center + point;
}
void mainImage(out vec4 fragColor, vec2 coord) {
vec2 uv = coord / iResolution.xy;
vec2 mouse = iMouse.xy / iResolution.xy;
uv = pinch(uv, mouse, PINCH_VECTOR, RANGE);
vec3 color = texture(iChannel0, uv).rgb;
fragColor.rgb = color;
}Summary
This article mainly introduces the realization principle of three types of local deformation Shader, in which the expansion/contraction and extrusion effects are realized by sampling distance field transformation. The former transforms the size of the sampling circle, and the latter transforms the position of the circle.
In addition to the three local deformations introduced above, there are some more interesting global deformation effects, such as wave effects, dislocation effects, mirroring, etc. Shader implementation is relatively easy, so I won’t introduce them further.
Wave-Displacement-Mirror
Preview code and effects
Distort: https://www.shadertoy.com/view/slfGzN
Expansion/Scale: https://www.shadertoy.com/view/7lXGzN
Extrusion/Stretch: https://www.shadertoy.com/view/7tX3zN
Reference material:
GLSL basic transformation: https://thebookofshaders.com/08/?lan=ch
Photoshop extrusion effect algorithm: https://blog.csdn.net/kezunhai/
That’s all for today’s sharing. Of course, life is boundless but knowing is boundless. In the long development cycle, these problems you see maybe just the tip of the iceberg. We have already prepared more technical topics on the UWA Q&A website, waiting for you to explore and share them together. You are welcome to join us, who love progress. Maybe your method can solve the urgent needs of others, and the “stone” of other mountains can also attack your “jade”.
YOU MAY ALSO LIKE!!!
UWA Website: https://en.uwa4d.com
UWA Blogs: https://blog.en.uwa4d.com
UWA Product: https://en.uwa4d.com/feature/got
You may also like
-
Dynamic Resolution in Games from Principle to Application
January 4, 2023 -
Introduction of the Raytracing Technology Part 2
December 21, 2022
