Poseestimation custom model returns empty keypoints

Hi team ,
I created a custom model to detect hand key point as mention in demo and tried to implement on poseestimation Android app available from fritz examples. but getting null keypoints exception

SDK version:latest
Changes to poseestimationapp :
1 :added custom model to asset directory
2:// Create a file called CustomSkeleton.java
public class CustomSkeleton extends Skeleton {
public static String OBJECT_NAME = “Some Object”;

public static string[] KEYPOINT_NAMES = {
“your”,
“keypoint”,
“names”,
“here”,
}

public CustomSkeleton() {
super(OBJECT_NAME, KEYPOINT_NAMES)
}
}

// Create a file called PoseModel.java
PoseOnDeviceModel onDeviceModel = new PoseOnDeviceModel(
“file:///android_asset/HandSyntheticModelFast1609957627.tflite”,
“f3ca25e573b144ac8c44e56c526e101c”,
2, // model version number
new CustomSkeleton(),
outputStride, // typically 8
false // required for custom models
);

2021-01-07 21:49:05.381 10398-10555/ai.fritz.poseestimationdemo E/RenderScript: Unable to open shared library (/data/user_de/0/ai.fritz.poseestimationdemo/code_cache/com.android.renderscript.cache/librs.rotator.so): (null)
2021-01-07 21:49:08.223 10398-10398/ai.fritz.poseestimationdemo E/AndroidRuntime: FATAL EXCEPTION: main
Process: ai.fritz.poseestimationdemo, PID: 10398
java.lang.NullPointerException: Attempt to get length of null array
at ai.fritz.vision.poseestimation.Pose.getConnectedKeypoints(Pose.java:110)
at ai.fritz.vision.poseestimation.Pose.draw(Pose.java:134)
at ai.fritz.camera.MainActivity.lambda$onPreviewSizeChosen$0$MainActivity(MainActivity.java:113)
at ai.fritz.camera.-$$Lambda$MainActivity$M2xvSBfN0iDRz7cLpyXc62MHAb4.drawCallback(Unknown Source:2)
at ai.fritz.camera.OverlayView.draw(OverlayView.java:33)
at android.view.View.updateDisplayListIfDirty(View.java:20302)
at android.view.ViewGroup.recreateChildDisplayList(ViewGroup.java:4372)
at android.view.ViewGroup.dispatchGetDisplayList(ViewGroup.java:4345)
at android.view.View.updateDisplayListIfDirty(View.java:20262)
at android.view.ViewGroup.recreateChildDisplayList(ViewGroup.java:4372)
at android.view.ViewGroup.dispatchGetDisplayList(ViewGroup.java:4345)
at android.view.View.updateDisplayListIfDirty(View.java:20262)
at android.view.ViewGroup.recreateChildDisplayList(ViewGroup.java:4372)
at android.view.ViewGroup.dispatchGetDisplayList(ViewGroup.java:4345)
at android.view.View.updateDisplayListIfDirty(View.java:20262)
at android.view.ViewGroup.recreateChildDisplayList(ViewGroup.java:4372)
at android.view.ViewGroup.dispatchGetDisplayList(ViewGroup.java:4345)
at android.view.View.updateDisplayListIfDirty(View.java:20262)
at android.view.ViewGroup.recreateChildDisplayList(ViewGroup.java:4372)
at android.view.ViewGroup.dispatchGetDisplayList(ViewGroup.java:4345)
at android.view.View.updateDisplayListIfDirty(View.java:20262)
at android.view.ViewGroup.recreateChildDisplayList(ViewGroup.java:4372)
at android.view.ViewGroup.dispatchGetDisplayList(ViewGroup.java:4345)
at android.view.View.updateDisplayListIfDirty(View.java:20262)
at android.view.ViewGroup.recreateChildDisplayList(ViewGroup.java:4372)
at android.view.ViewGroup.dispatchGetDisplayList(ViewGroup.java:4345)
at android.view.View.updateDisplayListIfDirty(View.java:20262)
at android.view.ViewGroup.recreateChildDisplayList(ViewGroup.java:4372)
at android.view.ViewGroup.dispatchGetDisplayList(ViewGroup.java:4345)
at android.view.View.updateDisplayListIfDirty(View.java:20262)
at android.view.ViewGroup.recreateChildDisplayList(ViewGroup.java:4372)
at android.view.ViewGroup.dispatchGetDisplayList(ViewGroup.java:4345)
at android.view.View.updateDisplayListIfDirty(View.java:20262)
at android.view.ThreadedRenderer.updateViewTreeDisplayList(ThreadedRenderer.java:575)
at android.view.ThreadedRenderer.updateRootDisplayList(ThreadedRenderer.java:581)
at android.view.ThreadedRenderer.draw(ThreadedRenderer.java:654)
at android.view.ViewRootImpl.draw(ViewRootImpl.java:3611)
at android.view.ViewRootImpl.performDraw(ViewRootImpl.java:3419)
at android.view.ViewRootImpl.performTraversals(ViewRootImpl.java:2756)
at android.view.ViewRootImpl.doTraversal(ViewRootImpl.java:1722)
at android.view.ViewRootImpl$TraversalRunnable.run(ViewRootImpl.java:7605)
at android.view.Choreographer$CallbackRecord.run(Choreographer.java:1041)
at android.view.Choreographer.doCallbacks(Choreographer.java:864)
at android.view.Choreographer.doFrame(Choreographer.java:799)
at android.view.Choreographer$FrameDisplayEventReceiver.run(Choreographer.java:1026)
at android.os.Handler.handleCallback(Handler.java:883)
at android.os.Handler.dispatchMessage(Handler.java:100)
at android.os.Looper.loop(Looper.java:214)
at android.app.ActivityThread.main(ActivityThread.java:7397)
at java.lang.reflect.Method.invoke(Native Method)
at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:492)
at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:935)

It does not appear that you have created a custom skeleton to match your model. The code template provided is not meant to be used directly. For example, you need to customize the keypoint names to match the model you trained. Please read the following instructions for more information: https://docs.fritz.ai/develop/vision/pose-estimation#3-define-a-skeleton

this how my skeleton file look like:

import ai.fritz.vision.poseestimation.Skeleton;

public class CustomSkeleton extends Skeleton {
public static String OBJECT_NAME = “hand”;

public static String[] KEYPOINT_NAMES = {
        "thumb",
        "index",
        "middle",
        "ring",
        "pinky"
};

public CustomSkeleton() {
    super(OBJECT_NAME, KEYPOINT_NAMES);
}

}

Does this example app build and run for you?

I want to make sure the problem is with your specific skeleton implementation and not some deeper issue. The example app linked above uses the exact same drawing code.

for this Im getting followin errors:

021-01-07 23:01:08.776 21111-21111/? E/eestimationdem: Unknown bits set in runtime_flags: 0x8000
2021-01-07 23:01:09.044 21111-21181/ai.fritz.poseestimationdemo E/Perf: Fail to get file list ai.fritz.poseestimationdemo
2021-01-07 23:01:09.044 21111-21181/ai.fritz.poseestimationdemo E/Perf: getFolderSize() : Exception_1 = java.lang.NullPointerException: Attempt to get length of null array
2021-01-07 23:01:09.044 21111-21181/ai.fritz.poseestimationdemo E/Perf: Fail to get file list ai.fritz.poseestimationdemo
2021-01-07 23:01:09.045 21111-21181/ai.fritz.poseestimationdemo E/Perf: getFolderSize() : Exception_1 = java.lang.NullPointerException: Attempt to get length of null array
2021-01-07 23:01:09.267 21111-21111/ai.fritz.poseestimationdemo E/libc: Access denied finding property “vendor.camera.aux.packagelist”
2021-01-07 23:01:09.267 21111-21111/ai.fritz.poseestimationdemo E/libc: Access denied finding property “vendor.camera.aux.packagelist”
2021-01-07 23:01:09.267 21111-21111/ai.fritz.poseestimationdemo E/libc: Access denied finding property “vendor.camera.aux.packagelist”
2021-01-07 23:01:09.267 21111-21111/ai.fritz.poseestimationdemo E/libc: Access denied finding property “vendor.camera.aux.packagelist”
2021-01-07 23:01:09.268 21111-21148/ai.fritz.poseestimationdemo E/libc: Access denied finding property “vendor.camera.aux.packagelist”
2021-01-07 23:01:09.268 21111-21148/ai.fritz.poseestimationdemo E/libc: Access denied finding property “vendor.camera.aux.packagelist”
2021-01-07 23:01:09.268 21111-21148/ai.fritz.poseestimationdemo E/libc: Access denied finding property “vendor.camera.aux.packagelist”
2021-01-07 23:01:09.475 21111-21111/ai.fritz.poseestimationdemo E/libc: Access denied finding property “persist.vendor.camera.privapp.list”
2021-01-07 23:01:09.480 21111-21148/ai.fritz.poseestimationdemo E/libc: Access denied finding property “vendor.camera.aux.packagelist”
2021-01-07 23:01:09.481 21111-21148/ai.fritz.poseestimationdemo E/libc: Access denied finding property “vendor.camera.aux.packagelist”
2021-01-07 23:01:10.052 21111-21287/ai.fritz.poseestimationdemo E/RenderScript: Unable to open shared library (/data/user_de/0/ai.fritz.poseestimationdemo/code_cache/com.android.renderscript.cache/librs.rotator.so): (null)
2021-01-07 23:01:12.055 21111-21111/ai.fritz.poseestimationdemo E/FritzCustomModelService: Job failed…rescheduling.
2021-01-07 23:01:22.623 21111-21111/ai.fritz.poseestimationdemo E/FritzCustomModelService: Job failed…rescheduling.
2021-01-07 23:01:43.100 21111-21111/ai.fritz.poseestimationdemo E/FritzCustomModelService: Job failed…rescheduling.
2021-01-07 23:02:19.550 21111-21136/ai.fritz.poseestimationdemo E/libc: Access denied finding property “vendor.camera.aux.packagelist”
2021-01-07 23:02:19.556 21111-21136/ai.fritz.poseestimationdemo E/libc: Access denied finding property “vendor.camera.aux.packagelist”
2021-01-07 23:02:23.735 21111-21111/ai.fritz.poseestimationdemo E/FritzCustomModelService: Job failed…rescheduling.

It seems like the error is related to your specific device and camera permissions there. What device are you building this on?

Im trying on Nokia 6.1 Plus (Global)
camera features:
Dual 16 MP, f/2.0, 1.0µm, PDAF
5 MP, f/2.4, (depth)
Features Dual-LED dual-tone flash,

I believe this is a permission problem and not a problem with the Fritz SDK:

Not sure if its camera issue, but when I tested fritz-examples\Android\FritzAIStudio this app runs well and also estimated pose (for default model)…the issue comes for this example app “fritz-examples\Android\PoseEstimationApp”

Interesting. Fritz AI studio and the pose estimation demo app all use the same code and same renderscript to draw the pose which is why I suspect this is related to some other configuration.