[Unreal] Real-time motion capture (IMU) data retarget to Unreal

W - Go Forward / S - Go Backward

A - Move Left / D - Move Right

---

https://www.patreon.com/posts/64346879

 

 

 

 

UDP Controller: access the human rigid body pars based on the sensor name > manipulate the rotational angels (rotation x, y, z | ptich, roll, yaw )

 

Add 3rd persion cahracter

content pack(add feature) > click add > 3rd person

 

3rd Person BluePrint: Event Graph

action defines: movement, jump, ...

All keys are associated for a specific control in the project settings > Engine > Input

-> bind all actions input based on controllers

 

 

+ Some animations: Mannequin > Animation folder

: created manually (by bone rotation every key frames)

---

Sensor Mapping to 3rd Person Character

Retarget: sensor details <-> human rigid body

 

- SensorTest_Human

- UDPController_Human (blueprint calss)

 

 

UDP controller blueprint

1. get the player character by calling fucn "get player character"

- multiple character: use player index to respective player character (default: 0)

2. get Anim Instance > cast To ThirdPerson

3. Switch case statement

- input: sensor name: ex_ upperarm_I / lowerarm_I

---

Project Folder

Mannequin

    |_Character

        |_Mesh: all the different skeletal mesh: male & female (can change character type)

 

    |_Bluprint

        |_instnace of the skeleton > properties > Mesh

 

---

NEXT...

 

1. SetUp: Unreal Engine

 

CSV -> data read > character mapping: action check

 

(JSON format)

 

 

 

 

 

● Using asynchronous UDP protocol: transmit the data to Unreal Engine.

● Establishing an UDP receiver in Unreal Engine. <- capture data: transmitted from sensor

 

 

Strign GetJSONString(String sensorName, float pitch, float roll, float yaw)
{
	StaticJsonDocument<1024> staticJsonDocument;
    StaticJsonDocument["name"] = sensorName;
    StaticJsonDocument["pitch"] = ptich;
    StaticJsonDocument["roll"] = roll;
    StaticJsonDocument["yaw"] = yaw;
    
    char docBuf[1024];
    serializeJson(staticJsonDocument, docBuf);
    
    return String(docBuf);
}

 

 

 

2. Unreal Engine: How to read csv file

 

- FFileHelper class

- FString class

 

 

❍ Create Data Table (CSV)

[참고1]

 

구조체 생성 시, USTURCT, GENRATE_BODY() 매크로 사용해줘야 Editor interface 에서 볼 수 있다.

CSV 파일에서 1열에 있는 데이터 -> 언리얼 엔진에서 자동으로 key  값으로 사용

 

❍ Import CSV file

구조체에 연결 후 적용

 

❍ Load Data Table

data에 따라.. 보통 변하지 않는 데이터는 게임 앱 초기화 할 때 불러들여 사용

 

 

 

---

[Reference]

 

 

https://www.youtube.com/watch?v=AbpgH0mMe34 

https://www.patreon.com/posts/real-time-motion-64976098