FreeMoCap

Allgemeines

Ich bin auf der Suche nach einer Möglichkeit meinen NAO als physischen Avatar zu benutzen, auf FreeMoCap gestoszen. Da es kostenlos ist und sich kinderleicht installieren liesz, habe ich es mal ausprobiert.

Die erste Begegnung galt der dem Programm. 2 Empfehlungen bin ich dabei gefolgt.

• Den ersten Versuch sollte man mit nur einer Kamera machen. Dabei entfaellt naemlich die Kalibrierung.
• Das ChArUco-Board solte grosz genug sein, dass es auch bei groeszerer Entfernung noch gut detektiert werden kann.

ChArUco

ChAruco ist dabei eine Variante bei der die ArUco-Marker (Binary Code Pattern) schachbrettfoermig (Chessboard) angeordnet sind.

An ArUco marker is a synthetic square marker composed by a wide black border and an inner binary matrix which determines its identifier (id). The black border facilitates its fast detection in the image and the binary codification allows its identification and the application of error detection and correction techniques. The marker size determines the size of the internal matrix. For instance a marker size of 4x4 is composed by 16 bits.

Quelle: [1]

Tutorial ArUco Detection

Automatic generation and detection of highly reliable fiducial markers under occlusion Augmented Reality University of Cordoba

Rumprobieren

Es gibt 2 Demodateien unter dem Menuepunkt "Data".

• Sample Data (3 cameras,~1000 frames)
• Test Data (3 cameras,~200 frames)

[2024-01-21T09:31:38.0205][INFO ] [ProcessID: 21325, ThreadID:139872446703360] [freemocap.utilities.download_sample_data:download_sample_data():30] ::: Downloading sample data from https://figshare.com/ndownloader/files/41368323.... [2024-01-21T09:31:38.0210][DEBUG ] [ProcessID: 21325, ThreadID:139872446703360] [urllib3.connectionpool:_new_conn():1052] ::: Starting new HTTPS connection (1): figshare.com:443. [2024-01-21T09:31:38.0448][DEBUG ] [ProcessID: 21325, ThreadID:139872446703360] [urllib3.connectionpool:_make_request():546] ::: https://figshare.com:443 "GET /ndownloader/files/41368323 HTTP/1.1" 302 0. [2024-01-21T09:31:38.0459][DEBUG ] [ProcessID: 21325, ThreadID:139872446703360] [urllib3.connectionpool:_new_conn():1052] ::: Starting new HTTPS connection (1): s3-eu-west-1.amazonaws.com:443. [2024-01-21T09:31:38.0769][DEBUG ] [ProcessID: 21325, ThreadID:139872446703360] [urllib3.connectionpool:_make_request():546] ::: https://s3-eu-west-1.amazonaws.com:443 "GET /pfigshare-u-files/41368323/freemocap_sample_data.zip?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAIYCQYOYV5JSSROOA/20240121/eu-west-1/s3/aws4_request&X-Amz-Date=20240121T083138Z&X-Amz-Expires=10&X-Amz-SignedHeaders=host&X-Amz-Signature=130140a8b4f41abda38f2fccb57cee17945bf25868cf44f5b09757fc7f6827ec HTTP/1.1" 200 166850523. [2024-01-21T09:32:43.0999][INFO ] [ProcessID: 21325, ThreadID:139872446703360] [freemocap.utilities.download_sample_data:download_sample_data():42] ::: Sample data extracted to /home/chris/freemocap_data/recording_sessions/freemocap_sample_data. [2024-01-21T09:32:44.0023][INFO ] [ProcessID: 21325, ThreadID:139872446703360] [freemocap.gui.qt.workers.download_sample_data_thread_worker:run():26] ::: Data successfully downloaded from: https://figshare.com/ndownloader/files/41368323. [2024-01-21T09:32:44.0033][INFO ] [ProcessID: 21325, ThreadID:139875417855808] [freemocap.gui.qt.main_window.freemocap_main_window:_handle_download_data_finished():508] ::: Setting downloaded data as active recording... . [2024-01-21T09:32:44.0034][INFO ] [ProcessID: 21325, ThreadID:139875417855808] [freemocap.gui.qt.widgets.active_recording_widget:set_active_recording():62] ::: Setting active recording to /home/chris/freemocap_data/recording_sessions/freemocap_sample_data. [2024-01-21T09:32:44.0035][DEBUG ] [ProcessID: 21325, ThreadID:139875417855808] [freemocap.gui.qt.widgets.active_recording_widget:_update_file_watch_path():78] ::: Updating file watch path to: /home/chris/freemocap_data/recording_sessions/freemocap_sample_data. [2024-01-21T09:32:44.0049][DEBUG ] [ProcessID: 21325, ThreadID:139875417855808] [freemocap.gui.qt.widgets.active_recording_widget:set_active_recording():68] ::: Emitting `new_active_recording_selected_signal` for recording: freemocap_sample_data. [2024-01-21T09:32:44.0050][INFO ] [ProcessID: 21325, ThreadID:139875417855808] [freemocap.gui.qt.main_window.freemocap_main_window:_handle_new_active_recording_selected():367] ::: New active recording selected: /home/chris/freemocap_data/recording_sessions/freemocap_sample_data. [2024-01-21T09:32:44.0374][INFO ] [ProcessID: 21325, ThreadID:139875417855808] [freemocap.data_layer.recording_models.recording_info_model:get_number_of_mp4s_synched_videos_directory():238] ::: Number of `.mp4`'s in /home/chris/freemocap_data/recording_sessions/freemocap_sample_data/synchronized_videos: 3.0. [2024-01-21T09:32:44.0576][INFO ] [ProcessID: 21325, ThreadID:139875417855808] [freemocap.gui.qt.widgets.directory_view_widget:set_folder_as_root():79] ::: Setting root folder to /home/chris/freemocap_data/recording_sessions/freemocap_sample_data. [2024-01-21T09:32:44.0583][DEBUG ] [ProcessID: 21325, ThreadID:139875417855808] [freemocap.gui.qt.widgets.directory_view_widget:expand_directory_to_path():63] ::: Collapsing other directories. [2024-01-21T09:32:44.0584][DEBUG ] [ProcessID: 21325, ThreadID:139875417855808] [freemocap.gui.qt.widgets.directory_view_widget:expand_directory_to_path():65] ::: Expanding directory at path: /home/chris/freemocap_data/recording_sessions/freemocap_sample_data/synchronized_videos. [2024-01-21T09:32:44.0584][DEBUG ] [ProcessID: 21325, ThreadID:139875417855808] [freemocap.gui.qt.widgets.directory_view_widget:expand_directory_to_path():73] ::: Expanding parent directory at path: /home/chris/freemocap_data/recording_sessions/freemocap_sample_data. [2024-01-21T09:32:44.0585][INFO ] [ProcessID: 21325, ThreadID:139875417855808] [freemocap.gui.qt.widgets.active_recording_widget:update_parameter_tree():74] ::: Updating Parameter Tree. [2024-01-21T09:32:44.0585][DEBUG ] [ProcessID: 21325, ThreadID:139875417855808] [freemocap.gui.qt.widgets.active_recording_widget:setup_parameter_tree():110] ::: Setting up `ActiveRecordingTreeView` for recording: freemocap_sample_data. [2024-01-21T09:32:50.0968][DEBUG ] [ProcessID: 21325, ThreadID:139875417855808] [freemocap.gui.qt.widgets.directory_view_widget:expand_directory_to_path():63] ::: Collapsing other directories. [2024-01-21T09:32:50.0984][DEBUG ] [ProcessID: 21325, ThreadID:139875417855808] [freemocap.gui.qt.widgets.directory_view_widget:expand_directory_to_path():65] ::: Expanding directory at path: /home/chris/freemocap_data/recording_sessions/freemocap_sample_data/annotated_videos. [2024-01-21T09:32:50.0984][DEBUG ] [ProcessID: 21325, ThreadID:139875417855808] [freemocap.gui.qt.widgets.directory_view_widget:expand_directory_to_path():73] ::: Expanding parent directory at path: /home/chris/freemocap_data/recording_sessions/freemocap_sample_data. [2024-01-21T09:32:51.0031][INFO ] [ProcessID: 21325, ThreadID:139875417855808] [freemocap.gui.qt.widgets.control_panel.calibration_control_panel:update_calibration_toml_path():127] ::: Setting calibration TOML path: /home/chris/freemocap_data/logs_info_and_settings/last_successful_calibration.toml. [2024-01-21T09:32:51.0040][INFO ] [ProcessID: 21325, ThreadID:139875417855808] [freemocap.gui.qt.widgets.control_panel.calibration_control_panel:update_calibration_toml_path():127] ::: Setting calibration TOML path: /home/chris/freemocap_data/logs_info_and_settings/last_successful_calibration.toml. [2024-01-21T09:32:51.0041][INFO ] [ProcessID: 21325, ThreadID:139875417855808] [freemocap.gui.qt.utilities.update_most_recent_recording_toml:update_most_recent_recording_toml():16] ::: Saving most recent recording path /home/chris/freemocap_data/recording_sessions/freemocap_sample_data to toml file: /home/chris/freemocap_data/logs_info_and_settings/most_recent_recording.toml. [2024-01-21T09:32:51.0163][INFO ] [ProcessID: 21325, ThreadID:139875417855808] [freemocap.data_layer.recording_models.recording_info_model:get_number_of_mp4s_synched_videos_directory():238] ::: Number of `.mp4`'s in /home/chris/freemocap_data/recording_sessions/freemocap_sample_data/synchronized_videos: 3.0.

1. Fehler

Im Reiter "Data Viewer" kann man nach erfolgreichem Download das Video abspielen, oder auch nicht.:-(

Im Reiter "Directory View" befindet sich folgender Eintrag: 2022-09-19_16_16_50_class_jsm_camera_calibration.toml

Fehlermeldung: 2024-01-21 09:46:47,140 - freemocap.gui.qt.widgets.directory_view_widget - INFO - Opening file from file_system_view_widget: /home/chris/freemocap_data/recording_sessions/freemocap_sample_data/2022-09-19_16_16_50_in_class_jsm_camera_calibration.toml QObject::moveToThread: Current thread (0x55bbe6b6f200) is not the object's thread (0x55bbe6b8db20). Cannot move to target thread (0x55bbe6b6f200)

qt.qpa.plugin: Could not load the Qt platform plugin "xcb" in "/usr/lib64/python3.11/site-packages/cv2/qt/plugins" even though it was found. This application failed to start because no Qt platform plugin could be initialized. Reinstalling the application may fix this problem.

Available platform plugins are: xcb, eglfs, linuxfb, minimal, minimalegl, offscreen, vnc, wayland-egl, wayland, wayland-xcomposite-egl, wayland-xcomposite-glx.

Da ich mir nicht schluessig war, wo der Fehler lag, habe ich die SampleFiles (freemocap_sample_data.zip) per Browser heruntergeladen (NB: Ein Versuch mit wget ergab Error 403: Fobidden). Es gab 2 Suchtreffer. Eine Datei mit 36,1 MB und eine mit 159 MB.

Auch ein nachfolgender Import brachte keine Erfolg.

Im Reiter "Data Viewer" steckt eigentlich der Skelly Viewer [2] dahinter. Beim Laden des Videoverzeichnisses zeigte sich folgender Fehler. Traceback (most recent call last):

 File "/usr/lib/python3.11/site-packages/skelly_viewer/gui/qt/skellyview_main_window.py", line 45, in _open_session_folder_dialog
   self._load_data(path=folder_path)
 File "/usr/lib/python3.11/site-packages/skelly_viewer/gui/qt/skellyview_main_window.py", line 58, in _load_data
   mediapipe_skeleton_npy_path=data_loader.find_skeleton_npy_file_name(),
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 File "/usr/lib/python3.11/site-packages/skelly_viewer/utilities/freemocap_data_loader.py", line 42, in find_skeleton_npy_file_name
   raise Exception(f"Could not find a skeleton NPY file in path {str(self.find_output_data_folder_path())}")

Exception: Could not find a skeleton NPY file in path /home/chris/Downloads/FreeMoCap_sample_klein/freemocap_sample_data/output_data

Um das NPY File zu erzeugen, bin mal noch einen Umweg uber den NPYViewer gegangen. Dabei habe ich auch noch gleich gh (Github CLI) installiert. Weiterhin habe ich festgestellt, dass mein openSuSE blender Version 2.8 voellig veraltet ist und mal blender v4 installiert.

Nochwas zum Probieren

[3]