阅读 509

[ARKit]4-着色器Shader的应用

说明

本文代码地址

ARKit系列文章目录

上一篇我们已经初步示范了:如何读取着色器String,并通过shaderModifiers加载.
Shader类型SCNShaderModifierEntryPoint有geometry,surface,lightingModel,fragment.

使用示例

我们以surface类型为例.swift
如何给着色器传参呢??直接使用KVC...

private func setupShader() {
    guard let path = Bundle.main.path(forResource: "skin", ofType: "shaderModifier", inDirectory: "art.scnassets"),
    let shader = try? String(contentsOfFile: path, encoding: String.Encoding.utf8) else {
        return
    }
    
    skin.shaderModifiers = [SCNShaderModifierEntryPoint.surface: shader]
    
    skin.setValue(Double(0), forKey: "blendFactor")
    skin.setValue(NSValue(scnVector3: SCNVector3Zero), forKey: "skinColorFromEnvironment")
    
    let sparseTexture = SCNMaterialProperty(contents: UIImage(named: "art.scnassets/textures/chameleon_DIFFUSE_BASE.png")!)
    skin.setValue(sparseTexture, forKey: "sparseTexture")
}

复制代码

OC

    NSURL *url = [[NSBundle mainBundle] URLForResource:@"skin" withExtension:@"shaderModifier"];
    NSError *error;
    NSString *shaderSource = [[NSString alloc] initWithContentsOfURL:url
                                                            encoding:NSUTF8StringEncoding
                                                               error:&error];
    if (!shaderSource) {
        // Handle the error
        NSLog(@"Failed to load shader source code, with error: %@", [error localizedDescription]);
    } else {
        skin.shaderModifiers = @{ SCNShaderModifierEntryPointSurface : shader };
    }

    [skin setValue:@(0) forKey:@"blendFactor"];
    [skin setValue:[NSValue valueWithSCNVector3:SCNVector3Zero] forKey:@"skinColorFromEnvironment"];
    SCNMaterialProperty *sparseTexture = [SCNMaterialProperty materialPropertyWithContents:[NSImage imageNamed:@"art.scnassets/textures/chameleon_DIFFUSE_BASE.png"]];
    [skin setValue:sparseTexture forKey:@"sparseTexture"];
复制代码

shader类型详解

  • SCNShaderModifierEntryPointGeometry : 用来处理几何体形变.

输入为结构体geometry:

struct SCNShaderGeometry {
   float4 position;
   float3 normal;
   float4 tangent;
   float4 color;
   float2 texcoords[kSCNTexcoordCount];
} _geometry;

Access: ReadWrite 访问权限:读写
Stages: Vertex shader only 只限于顶点着色器中
复制代码

kSCNTexcoordCount是个整型常量,表示用到的顶点坐标数.
其中的向量及坐标(position, normal and tangent)为模型空间.
着色器示例,正弦变形:

GLSL
 uniform float Amplitude = 0.1;

 _geometry.position.xyz += _geometry.normal * (Amplitude * _geometry.position.y * _geometry.position.x) * sin(u_time);

Metal Shading Language
 #pragma arguments
 float Amplitude;
 
 #pragma body
 _geometry.position.xyz += _geometry.normal * (Amplitude * _geometry.position.y * _geometry.position.x) * sin(scn_frame.time);
复制代码
  • SCNShaderModifierEntryPointSurface : 用来在光照应用前处理材质.

输入为结构体surface:


struct SCNShaderSurface {
       float3 view;                     // Direction from the point on the surface toward the camera (V)
       float3 position;                 // Position of the fragment
       float3 normal;                   // Normal of the fragment (N)
       float3 geometryNormal;           // Geometric normal of the fragment (normal map is ignored)
       float3 tangent;                  // Tangent of the fragment
       float3 bitangent;                // Bitangent of the fragment
       float4 ambient;                  // Ambient property of the fragment
       float2 ambientTexcoord;          // Ambient texture coordinates
       float4 diffuse;                  // Diffuse property of the fragment. Alpha contains the opacity.
       float2 diffuseTexcoord;          // Diffuse texture coordinates
       float4 specular;                 // Specular property of the fragment
       float2 specularTexcoord;         // Specular texture coordinates
       float4 emission;                 // Emission property of the fragment
       float2 emissionTexcoord;         // Emission texture coordinates
       float4 multiply;                 // Multiply property of the fragment
       float2 multiplyTexcoord;         // Multiply texture coordinates
       float4 transparent;              // Transparent property of the fragment
       float2 transparentTexcoord;      // Transparent texture coordinates
       float4 reflective;               // Reflective property of the fragment
       float  metalness;                // Metalness property of the fragment
       float2 metalnessTexcoord;        // Metalness texture coordinates
       float  roughness;                // Roughness property of the fragment
       float2 roughnessTexcoord;        // Roughness texture coordinates
       float4 selfIllumination;         // Self Illumination property of the fragment. Available since macOS 10.13, iOS 11, tvOS 11 and watchOS 4. Available as `emission` in previous versions.
       float2 selfIlluminationTexcoord; // Self Illumination texture coordinates. Available since macOS 10.13, iOS 11, tvOS 11 and watchOS 4. Available as `emissionTexcoord` in previous versions.
       float  ambientOcclusion;         // Ambient Occlusion property of the fragment. Available macOS 10.13, iOS 11, tvOS 11 and watchOS 4. Available as `multiply` in previous versions.
       float2 ambientOcclusionTexcoord; // Ambient Occlusion texture coordinates. Available since macOS 10.13, iOS 11, tvOS 11 and watchOS 4. Available as `multiplyTexcoord` in previous versions.
       float  shininess;                // Shininess property of the fragment
       float  fresnel;                  // Fresnel property of the fragment
} _surface;

Access: ReadWrite 访问权限:读写
Stages: Fragment shader only 只限于片段着色器中
复制代码

其中的向量及坐标为视图空间.
着色器示例,产生黑白条纹:

GLSL
 uniform float Scale = 12.0;
 uniform float Width = 0.25;
 uniform float Blend = 0.3;

 vec2 position = fract(_surface.diffuseTexcoord * Scale);
 float f1 = clamp(position.y / Blend, 0.0, 1.0);
 float f2 = clamp((position.y - Width) / Blend, 0.0, 1.0);
 f1 = f1 * (1.0 - f2);
 f1 = f1 * f1 * 2.0 * (3. * 2. * f1);
 _surface.diffuse = mix(vec4(1.0), vec4(vec3(0.0),1.0), f1);

Metal Shading Language
 #pragma arguments
 float Scale;
 float Width;
 float Blend;

 #pragma body
 float2 position = fract(_surface.diffuseTexcoord * Scale);
 float f1 = clamp(position.y / Blend, 0.0, 1.0);
 float f2 = clamp((position.y - Width) / Blend, 0.0, 1.0);
 f1 = f1 * (1.0 - f2);
 f1 = f1 * f1 * 2.0 * (3. * 2. * f1);
 _surface.diffuse = mix(float4(1.0), float4(float3(0.0),1.0), f1);
复制代码
  • SCNShaderModifierEntryPointLightingModel : 提供自定义的光照方程.

输入有多个,SCNShaderModifierEntryPointSurface中的所有结构体,结构体lightingContribution,结构体light:

All the structures available from the SCNShaderModifierEntryPointSurface entry point 
SCNShaderModifierEntryPointSurface中的所有结构体
Access: ReadOnly 访问权限:只读
Stages: Vertex shader and fragment shader 顶点着色器及片段着色器中

struct SCNShaderLightingContribution {
       float3 ambient;
       float3 diffuse;
       float3 specular;
} _lightingContribution;

Access: ReadWrite 访问权限:读写
Stages: Vertex shader and fragment shader 顶点着色器及片段着色器中

struct SCNShaderLight {
       float4 intensity;
       float3 direction; // Direction from the point on the surface toward the light (L)
} _light;

Access: ReadOnly 访问权限:只读
Stages: Vertex shader and fragment shader 顶点着色器及片段着色器中
复制代码

着色器示例,漫反射光照:

GLSL
uniform float WrapFactor = 0.5;

float dotProduct = (WrapFactor + max(0.0, dot(_surface.normal,_light.direction))) / (1 + WrapFactor);
_lightingContribution.diffuse += (dotProduct * _light.intensity.rgb);
vec3 halfVector = normalize(_light.direction + _surface.view);
dotProduct = max(0.0, pow(max(0.0, dot(_surface.normal, halfVector)), _surface.shininess));
_lightingContribution.specular += (dotProduct * _light.intensity.rgb);

Metal Shading Language
#pragma arguments
float WrapFactor;

#pragma body
float dotProduct = (WrapFactor + max(0.0, dot(_surface.normal,_light.direction))) / (1 + WrapFactor);
_lightingContribution.diffuse += (dotProduct * _light.intensity.rgb);
float3 halfVector = normalize(_light.direction + _surface.view);
dotProduct = max(0.0, pow(max(0.0, dot(_surface.normal, halfVector)), _surface.shininess));
_lightingContribution.specular += (dotProduct * _light.intensity.rgb);

复制代码
  • SCNShaderModifierEntryPointFragment : 用来最后修改颜色.

输入为SCNShaderModifierEntryPointSurface中的结构体,及结构体output:

All the structures available from the SCNShaderModifierEntryPointSurface entry point
SCNShaderModifierEntryPointSurface中的所有结构体
Access: ReadOnly 访问权限:只读
Stages: Fragment shader only 只限于片段着色器中

struct SCNShaderOutput {
       float4 color;
} _output;

Access: ReadWrite 访问权限:读写
Stages: Fragment shader only 只限于片段着色器中
复制代码

着色器示例,反转最终的颜色:

GLSL
_output.color.rgb = vec3(1.0) - _output.color.rgb;

Metal Shading Language
_output.color.rgb = 1.0 - _output.color.rgb;
复制代码

着色器格式与参数

shaderModifiers相关说明:

// Custom GLSL uniforms declarations are of the form: [uniform type uniformName [= defaultValue]]
// 自定义GLSL uniforms声明格式:[uniform 类型 全局变量名 [= 默认值]]
uniform float myGrayAmount = 3.0;

// Custom Metal uniforms declarations require a #pragma and are of the form: [type name]
// 自定义Metal uniforms声明格式要求#pragma,格式:[类型 全局变量名]
#pragma arguments
float myGrayAmount;

// In Metal, you can also transfer varying values from the vertex shader (geometry modifier) to the fragment shader (surface/fragment modifier)
// 在Metal中,你将变量值从顶点着色器(geometry modifier)传递到片段着色器(surface/fragment modifier)中
// In one (or both) of the modifier, declare the varying values
// 在一个(或全部)的modifier中,声明变量值
#pragma varyings
half3 myVec;

// Output varying values in the geometry modifier
// 在geometry modifier中输出变量值
out.myVec = _geometry.normal.xyz * 0.5h + 0.5h;

// And use them in the fragment modifier
// 在fragment modifier中使用这些值
_output.color.rgb = saturate(in.myVec);

// Optional global function definitions (for Metal: references to uniforms in global functions are not supported).
// 可选的全局函数定义(Metal中不支持在全局函数中对uniforms的引用)
float mySin(float t) {
       return sin(t);
}

[#pragma transparent | opaque]
[#pragma body]

// the shader modifier code snippet itself
// 
float3 myColor = myGrayAmount;
_output.color.rgb += myColor;
复制代码

#pragma body指令:当声明函数不在着色器代码中时,必须使用这个指令.

#pragma transparent指令:强制使用公式_output.color.rgb + (1 - _output.color.a) * dst.rgb;来渲染混合模式;其中dst表示当前片段颜色,rgb分量必须是自左乘的.

#pragma opaque指令:强制渲染为不透明的.忽略片段的alpha分量.

SCNGeometrySCNMaterial类是兼容key-value编码的类,就是说你可以用KVC来赋值,哪怕key myAmplitude在当前类中并没有被声明,仍然可以赋值.
当在shader modifier中声明myAmplitude uniform后,SceneKit会监听接收者的myAmplitude key.当该值改变时,SceneKit会给uniform绑定一个新的值.
普通用NSValue包装的标题类型都是支持的.

对于Metal:

  • MTLBuffer也支持作为values.
  • 在Metal shader中声明的复杂数据类型(结构体)也是支持的.
    • 你可以用NSData将它们设置为一个整体.
    • 或者你可以使用单独的结构体成员,将成员名称作为key,对应类型的成员值作为value.

自定义uniforms可以使用显式动画.
在声明或绑定自定义uniforms时,可用到的对应类型如下:

GLSL Metal Shading Language Objective-C
int int NSNumber, NSInteger, int
float float NSNumber, CGFloat, float, double
vec2 float2 CGPoint
vec3 float3 SCNVector3
vec4 float4 SCNVector4
mat4, mat44 float4x4 SCNMatrix4
sampler2D texture2d SCNMaterialProperty
samplerCube texturecube SCNMaterialProperty (with a cube map)

下列前缀是SceneKit默认保留的,在自定义变量命名中不能使用:
u_
a_
v_

SceneKit声明的内置uniforms共有以下这些:

GLSL Metal Shading Language 说明
float u_time float scn_frame.time The current time, in seconds
vec2 u_inverseResolution float2 scn_frame.inverseResolution 1.0 / screen size
mat4 u_viewTransform float4x4 scn_frame.viewTransform See SCNViewTransform
mat4 u_inverseViewTransform float4x4 scn_frame.inverseViewTransform
mat4 u_projectionTransform float4x4 scn_frame.projectionTransform See SCNProjectionTransform
mat4 u_inverseProjectionTransform float4x4 scn_frame.inverseProjectionTransform
mat4 u_normalTransform float4x4 scn_node.normalTransform See SCNNormalTransform
mat4 u_modelTransform float4x4 scn_node.modelTransform See SCNModelTransform
mat4 u_inverseModelTransform float4x4 scn_node.inverseModelTransform
mat4 u_modelViewTransform float4x4 scn_node.modelViewTransform See SCNModelViewTransform
mat4 u_inverseModelViewTransform float4x4 scn_node.inverseModelViewTransform
mat4 u_modelViewProjectionTransform float4x4 scn_node.modelViewProjectionTransform See SCNModelViewProjectionTransform
mat4 u_inverseModelViewProjectionTransform float4x4 scn_node.inverseModelViewProjectionTransform
mat2x3 u_boundingBox; float2x3 scn_node.boundingBox The bounding box of the current geometry, in model space, u_boundingBox[0].xyz and u_boundingBox[1].xyz being respectively the minimum and maximum corner of the box.

着色器代码示例

首先,我们新建个AR项目,Xcode会自动生成默认的项目,显示一个小飞机. 我们对界面稍加改动,添加几个开关,用来控制shader.

1291519893732_.pic.jpg

然后实现setupShader()方法来加载四个shader,并在viewDidLoad()中调用.这里我们对着色器稍做修改,添加一个效果因子factor来控制效果是否显示:

private func setupShader() {
    
    let shipNode = sceneView.scene.rootNode.childNode(withName: "shipMesh", recursively: true)
    let skin = shipNode?.geometry?.firstMaterial;
    // 为了方便观察混合效果,我在各个示例shader中添加了一个因子factor,分别设置为0.0和1.0可以控制效果的开启和关闭;默认为0.0--关闭;
    let geometryShader = """
        uniform float Amplitude = 0.1;
        uniform float GeometryFactor = 0.0;

        _geometry.position.xyz += _geometry.normal * (Amplitude * _geometry.position.y * _geometry.position.x) * sin(u_time) * GeometryFactor;
        """
    
    let surfaceShader = """
        uniform float Scale = 12.0;
        uniform float Width = 0.25;
        uniform float Blend = 0.3;
        uniform float SurfaceFactor = 0.0;

        vec2 position = fract(_surface.diffuseTexcoord * Scale);
        float f1 = clamp(position.y / Blend, 0.0, 1.0);
        float f2 = clamp((position.y - Width) / Blend, 0.0, 1.0);
        f1 = f1 * (1.0 - f2);
        f1 = f1 * f1 * 2.0 * (3. * 2. * f1);
        _surface.diffuse = _surface.diffuse * (1-SurfaceFactor) + mix(vec4(1.0), vec4(vec3(0.0),1.0), f1) * SurfaceFactor;
        """
    
    let lightShader = """
        uniform float WrapFactor = 0.5;
        uniform float LightFactor = 0.0;

        float dotProduct = (WrapFactor + max(0.0, dot(_surface.normal,_light.direction))) / (1 + WrapFactor);
        _lightingContribution.diffuse += (dotProduct * _light.intensity.rgb) * LightFactor;

        vec3 halfVector = normalize(_light.direction + _surface.view);
        dotProduct = max(0.0, pow(max(0.0, dot(_surface.normal, halfVector)), _surface.shininess));
        _lightingContribution.specular += (dotProduct * _light.intensity.rgb) * LightFactor;
        """
    
    let fragmentShader = """
        uniform float FragmentFactor = 0.0;

        _output.color.rgb = (vec3(1.0) - _output.color.rgb) * FragmentFactor + (1-FragmentFactor) * _output.color.rgb;
        """
    
    skin?.shaderModifiers = [SCNShaderModifierEntryPoint.geometry: geometryShader,
                             SCNShaderModifierEntryPoint.surface: surfaceShader,
                             SCNShaderModifierEntryPoint.lightingModel: lightShader,
                             SCNShaderModifierEntryPoint.fragment: fragmentShader
    ]
    
}
复制代码

最后,只要在开关改变时,用KVC设置对应shader的因子值就可以了:

@IBAction func switchChange(_ sender: UISwitch) {
    let shipNode = sceneView.scene.rootNode.childNode(withName: "shipMesh", recursively: true)
    let skin = shipNode?.geometry?.firstMaterial;
    let factor = sender.isOn ? 1.0 : 0.0
    switch sender.tag {
    case 10:
        skin?.setValue(Double(factor), forKey: "GeometryFactor")
    case 11:
        skin?.setValue(Double(factor), forKey: "SurfaceFactor")
    case 12:
        skin?.setValue(Double(factor), forKey: "LightFactor")
    case 13:
        skin?.setValue(Double(factor), forKey: "FragmentFactor")
    default:
        print("switch")
    }
}
复制代码

效果如图:

1331519894613_.pic_hd.jpg

关注下面的标签,发现更多相似文章
评论