Extend keyguard screen lock

In the last post, we discussed how to extend the screen lock settings for the face enrollment. This post will continue that and talks a bit further to the keyguard part. First, I want show you the sequence diagram of keyguard view management.

Keyguard view management sequence diagram

This diagram illustrates the whole picture of how keyguard is stated and the view is initialized. The first three classes are located at frameworks/base/policy/src/com/android/internal/policy/impl/keyguard of Android system source code. The KeyguardServiceDelegate class is the service interface which can communicate with the KeyguardServiceWrapper and direct requests from the clients that connect with the service. KeyguardServiceWrapper implements the IKeyguardService interface defined in frameworks/base/core/java/android/app/admin/IKeyguardService.aidl. It's the class that actually calls the service methods. The KeyguardService implementation is located at "frameworks/base/packages/Keyguard/src/com/android/keyguard/KeyguardService.java". It implements the methods in IKeyguardService and expose the services.
Now we are arriving at the KeyguardService implementation part. The source codes for KeyguardViewMediator, KeyguardViewManager, KeyguardHostView and KeyguardSecurityView are in the "frameworks/base/packages/Keyguard/src/com/android/keyguard" folder. We'll discuss them in detail.

1. KeyguardViewMediator
   This is the first class that deals with keyguard screen lock view. Like its name, KeyguardViewMediator is the scheduler of keyguard views and keep monitoring the state change of the phone. The doKeyguardLocked function of this class makes a list of checks about whether keyguard should show. At last it calls showLocked and invokes the show function of KeyguardViewManager.
2. KeyguardViewManager
   Move directly to the show function which is supposed to show the keyguard. There are three key objects in this function: mKeyguardHost of type ViewManagerHost, mViewManger of type ViewManger and mKeyguardView of type KeyguardHostView. The ViewMangerHost class is defined in KeyguardViewManger.java and is inherited from FrameLayout. It setup the layout parameters of the view and make the view visible. So where are mKeyguardView and mKeyguardHost initialized? It's right in the function maybeCreateKeyguardLocked. In that function, the mKeyguardHost is being created and added to the view group. The layout parameters are also set up here. Then it calls the inflateKeyguardView function to inflate the keyguard_host_view which is defined in "frameworks/base/packages/Keyguard/res/layout-land/keyguard_host_view.xml".  Then it calls the  show function of KeyguardHostView.
3. KeyguardHostView
   The control finally goes to the showSecurityScreen function of KeyguardHostView. It's in this function that we loaded the corresponding unlock view as we setup in the screen lock settings. It retrieves the current unlock view by the getSecurityView function based on current security mode. If the security view has not been added yet, it inflates the corresponding layout which is obtained by the getLayoutIdFor(securityMode) function. We added out face unlock entry in this function and define the layout as keyguard_biometric_face_view.xml in "frameworks/base/packages/Keyguard/res/layout". The source code for the view is "frameworks/base/packages/Keyguard/src/com/android/keyguard/KeyguardBiometric FaceView.java". It implements KeyguardSecurityView and SurfaceHolder.Callback to show camera preview.
4. KeyguardSecurityView
   This is the unlock view class we actually put our unlock mechanism in. There are two problems in this class we need to deal with:
• How to show camera preview
• How to communicate with the face recognition activity
When I search solution for the first problem, the most possible answer I could find is to add a SurfaceView container in the layout and add the SurfaceView in the keyguard view group. However, the camera would not show preview no matter how I initialize the SurfaceView. I realized it could be the keyguard window that forbids the preview of camera. Then I checked the original face unlock view of Android system,


  void handleServiceConnected() {
        Log.d(TAG, "handleServiceConnected()");

        // It is possible that an unbind has occurred in the time between the bind and when this
        // function is reached.  If an unbind has already occurred, proceeding on to call startUi()
        // can result in a fatal error.  Note that the onServiceConnected() callback is
        // asynchronous, so this possibility would still exist if we executed this directly in
        // onServiceConnected() rather than using a handler.
        if (!mBoundToService) {        
            Log.d(TAG, "Dropping startUi() in handleServiceConnected() because no longer bound");
            return;
        }

        try {
            mService.registerCallback(mFaceUnlockCallback);
        } catch (RemoteException e) {  
            Log.e(TAG, "Caught exception connecting to Face Unlock: " + e.toString());
            mService = null;
            mBoundToService = false;       
            mIsRunning = false;            
            return;
        }

        if (mFaceUnlockView != null) { 
            IBinder windowToken = mFaceUnlockView.getWindowToken();
            if (windowToken != null) {     
                // When switching between portrait and landscape view while Face Unlock is running,
                // the screen will eventually go dark unless we poke the wakelock when Face Unlock
                // is restarted.               
                mKeyguardScreenCallback.userActivity(0);

                int[] position;                
                position = new int[2];         
                mFaceUnlockView.getLocationInWindow(position);
                startUi(windowToken, position[0], position[1], mFaceUnlockView.getWidth(),
                        mFaceUnlockView.getHeight());
            } else {
                Log.e(TAG, "windowToken is null in handleServiceConnected()");
            }
        }
    }

I have an interesting finding. Since the face unlock code is proprietary from pittpatt which is now acquired by Google, we can not see the source code for the face unlock service. However, I get to know it uses the bound service to communicate with the face unlock model. And even more inspiring,  the startUi function exposes some implementation of the service. It passes the windowToken of keyguard window, the location and size of the face unlock view "place holder". Yes, I call it "place holder" since I believe it's only a place holder for the actual face unlock view which is added in the service implementation. So that in the face unlock view we can define the window parameters for the camera preview, which could solve the problem of camera preview won't show on keygurad window.
So the next question is, "is it possible to create new view element in the background service?".  Yes, it is possible to draw window in the background service. The facebook chatheads proves that according to this article http://www.piwai.info/chatheads-basics/. So we did exactly the same thing, draw a new window at the place holder with the same location and size and set the window type to be TYPE_SYSTEM_OVERLAY. And also we want to keep the screen on while conducting the face unlock, we can set the FLAG_KEEP_SCREEN_ON to the new window. Then add SurfaceView to this window, the camera preview will show up this time!
Now we are almost done unless for the face recognition service. I used two-way bound service for communication between the service and KeyguardSecurityView.

Acknowledgement:
Some of the content from this post refers the following websites:

•  http://www.programering.com/a/MDOyIDNwATQ.html
• http://www.cnblogs.com/haiming/p/2989678.html
• pittpatt proprietary files: https://github.com/jamesonwilliams/vendor_google_gapps/tree/android-4.4.2_r1/proprietary/optional/face/vendor/pittpatt/models/detection/multi_pose_face_landmark_detectors.7
•  facelock.apk: https://github.com/jamesonwilliams/vendor_google_gapps/find/android-4.4_r1.1/
• Include jar in Android.mk: http://www.cnblogs.com/hopetribe/archive/2012/04/23/2467060.html
• Chathead Basics:http://www.piwai.info/chatheads-basics/
•   Binder and Window Tokens: http://www.androiddesignpatterns.com/2013/07/binders-window-tokens.html