说明
本文是Ray Wenderlich上《ARKit by Tutorials》的读书笔记,主要讲内容概要和读后感
没错,本文主要讲iPhone X的前置TrueDepth摄像头的AR效果!主要功能:
- 面部检测和追踪
- 实时面部表情追踪
- 同时捕捉彩色图像,和深度图像
- 光照估计
初始化配置
面部AR的启动和其他的差不多,只需要将配置项换成ARFaceTrackingConfiguration()
就可以了,错误处理也类似:
func resetTracking() {
// 1
guard ARFaceTrackingConfiguration.isSupported else {
updateMessage(text: "Face Tracking Not Supported.")
return
}
// 2
updateMessage(text: "Looking for a face.")
// 3
let configuration = ARFaceTrackingConfiguration()
configuration.isLightEstimationEnabled = true /* default setting */
configuration.providesAudioData = false /* default setting */
// 4
session.run(configuration, options:
[.resetTracking, .removeExistingAnchors])
}
func session(_ session: ARSession, didFailWithError error: Error) {
print("** didFailWithError")
updateMessage(text: "Session failed.")
}
func sessionWasInterrupted(_ session: ARSession) {
print("** sessionWasInterrupted")
updateMessage(text: "Session interrupted.")
}
func sessionInterruptionEnded(_ session: ARSession) {
print("** sessionInterruptionEnded")
updateMessage(text: "Session interruption ended.")
}
人脸追踪
ARKit在检测到人脸时,会添加一个ARFaceAnchor
到场景中,我们就可以用这个锚点来实现定位和追踪功能.
如果有两张人脸,ARKit只会追踪最大,最具有辨识度的那张脸.同时会在人脸上添加一个人脸坐标系:
我们用scene的Metal device来创建人脸的几何体ARSCNFaceGeometry
,并为其创建一个SCNNode对象mask
来持有几何体:
func createFaceGeometry() {
updateMessage(text: "Creating face geometry.")
let device = sceneView.device!
let maskGeometry = ARSCNFaceGeometry(device: device)!
mask = Mask(geometry: maskGeometry)
}
这个方法可以在viewDidLoad
里面直接调用,无需等到人脸出现再创建.
最后将mask
添加到视图上以显示出来:
func renderer(_ renderer: SCNSceneRenderer, didAdd node: SCNNode, for anchor: ARAnchor) {
anchorNode = node
//将mask显示出来
setupFaceNodeContent()
}
func setupFaceNodeContent() {
guard let node = anchorNode else { return }
node.childNodes.forEach { $0.removeFromParentNode() }
if let content = mask {
node.addChildNode(content)
}
}
前面创建的几何体ARSCNFaceGeometry
因为还没有识别到人脸,可能是空的.所以我们需要在后面的update方法中更新.
func renderer(_ renderer: SCNSceneRenderer, didUpdate node: SCNNode, for anchor: ARAnchor) {
guard let faceAnchor = anchor as? ARFaceAnchor else { return }
updateMessage(text: "Tracking your face.")
// 根据锚点更新几何体
mask?.update(withFaceAnchor: faceAnchor)
}
// Tag: ARFaceAnchor Update
func update(withFaceAnchor anchor: ARFaceAnchor) {
let faceGeometry = geometry as! ARSCNFaceGeometry
faceGeometry.update(from: anchor.geometry)
}
默认情况下灯光设置是这样:
/* default settings */
sceneView.automaticallyUpdatesLighting = true
sceneView.autoenablesDefaultLighting = false
sceneView.scene.lightingEnvironment.intensity = 1.0
但我们也可以根据环境调整,以实现不同的效果:
func renderer(_ renderer: SCNSceneRenderer, updateAtTime time: TimeInterval) {
// 1
guard let estimate = session.currentFrame?.lightEstimate else {
return
}
// 2 在ARKit中1000意味着光强为中等
let intensity = estimate.ambientIntensity / 1000.0
sceneView.scene.lightingEnvironment.intensity = intensity
// 3
let intensityStr = String(format: "%.2f", intensity)
let sceneLighting = String(format: "%.2f",
sceneView.scene.lightingEnvironment.intensity)
// 4
print("Intensity: \(intensityStr) - \(sceneLighting)")
}
blend shapes表情动画
如何实现Animojis中的表情动画效果呢?这就用到了blend shapes,它本质是一个字典,key是ARFaceAnchor.BlendShapeLocation
常量,而value则是浮点数,范围0.0(自然状态)~1.0(最大移动状态).
比如,我们想添加一个眨眼的效果,取出eyeBlinkLeft
:
这个字典的是随着ARFaceAnchor更新而更新的:
// - Tag: ARFaceAnchor Update
func update(withFaceAnchor anchor: ARFaceAnchor) {
blendShapes = anchor.blendShapes
}
// - Tag: BlendShapeAnimation
var blendShapes: [ARFaceAnchor.BlendShapeLocation: Any] = [:] {
didSet {
guard
// Brow
let browInnerUp = blendShapes[.browInnerUp] as? Float
// Right eye
let eyeLookInRight = blendShapes[.eyeLookInRight] as? Float,
let eyeLookOutRight = blendShapes[.eyeLookOutRight] as? Float,
let eyeLookUpRight = blendShapes[.eyeLookUpRight] as? Float,
let eyeLookDownRight = blendShapes[.eyeLookDownRight] as? Float,
let eyeBlinkRight = blendShapes[.eyeBlinkRight] as? Float
else { return }
// 在处理动画
}
实现后的效果
ReplayKit录像
ReplayKit是在iOS 9引入的,用来录制音频,视频和麦克风,主要有两个类:
- RPScreenRecorder:用来启动/停止录制的单例.
- RPPreviewViewController:录制完成后的预览控制器.
- iOS屏幕录制和广播
- 广播配对
- 快速摄像头切换
- 应用内屏幕捕捉
开始录制
主要代码
let sharedRecorder = RPScreenRecorder.shared()
private var isRecording = false
// Private functions
private func startRecording() {
// 1
self.sharedRecorder.isMicrophoneEnabled = true
// 2
sharedRecorder.startRecording( handler: { error in
guard error == nil else {
print("There was an error starting the recording: \(String(describing: error?.localizedDescription))")
return
}
// 3
print("Started Recording Successfully")
self.isRecording = true
// 4
DispatchQueue.main.async {
self.recordButton.setTitle("[ STOP RECORDING ]", for: .normal)
self.recordButton.backgroundColor = UIColor.red
}
})
}
停止录制
主要代码
func stopRecording() {
// 1
self.sharedRecorder.isMicrophoneEnabled = false
// 2
sharedRecorder.stopRecording( handler: {
previewViewController, error in
guard error == nil else {
print("There was an error stopping the recording: \(String(describing: error?.localizedDescription))")
return
}
// 3
if let unwrappedPreview = previewViewController {
unwrappedPreview.previewControllerDelegate = self
self.present(unwrappedPreview, animated: true, completion: {})
}
})
// 4
self.isRecording = false
DispatchQueue.main.async {
self.recordButton.setTitle("[ RECORD ]", for: .normal)
self.recordButton.backgroundColor = UIColor(red: 0.0039,
green: 0.5882, blue: 1, alpha: 1.0) /* #0196ff */
}
}
效果如图:
另外,还可以实现sharedRecorder
的代理方法,监听错误回调和状态改变:
// RPScreenRecorderDelegate methods
func screenRecorder(_ screenRecorder: RPScreenRecorder,
didStopRecordingWith previewViewController: RPPreviewViewController?,
error: Error?) {
guard error == nil else {
print("There was an error recording: \(String(describing:
error?.localizedDescription))")
self.isRecording = false
return
}
}
func screenRecorderDidChangeAvailability(_ screenRecorder:
RPScreenRecorder) {
recordButton.isEnabled = sharedRecorder.isAvailable
if !recordButton.isEnabled {
self.isRecording = false
}
}
使用方法不算太难,此处不再过多展开,详情可购买阅读正版书籍.
第四部分读书笔记结束!