Development environment for controlling robot toys with a smartphone app "toio SDK for Unity"

Unity Technologies Japan's official online seminar "Unity Dojo Robotics Special" was held on October 9 and October 10, 2020.Unity Dojo is a study session for unity experts to teach all categories in a wide range of developers, from students to working people.The theme this time is Robotics.

Among them, we interviewed a session on "Toio SDK for Unity" to freely control the robot Toy "Toyo ™" of Sony Interactive Entertainment (hereinafter, SIE).The presenters are Shota Honma, an AI engineer from Moricatron Co., Ltd., who was involved in the design of "TOIO SDK for Unity", Shota Honma, who was involved in the design of "TOIO SDK for Unity".

Development environment that controls robot Toys with apps

"TOIO" is a cube -shaped robot toy aimed at bringing out the ingenuity of children.It is designed as a platform that allows you to enjoy free play that is not bound by crafts and programming by combining with various separate titles such as action games and puzzle games.

It is also used as a programming material according to a wide range of ages.For children in elementary school to junior high school, the basic structure of programming, such as "sequence, branching, and repetition", publishes javascript libraries and technical specifications to junior and senior high school students and engineers in the upper grades of elementary schools, and to junior and senior high school students and engineers.So, it provides a full -fledged robot programming opportunity.

Moricatron has developed "TOIO SDK for Unity" to link this "TOIO" to the unity of the game engine.This allows you to reproduce the core cube of "toio" on the simulator, and to generate applications to control the cube via a smart device.In addition, depending on the library and assets, AR and machine learning can be integrated.You can expect not only robot engineering but also media art.

Not only single control but also group control of multiple individuals is possible

The "TOIO SDK for Unity" includes the "Cube class" that programs the cube, the "Bluetooth" that controls the device, the "Simulator", which simulates the behavior of "toio" in Unityeditor, and the cube as a single or a crowd.There are four types of modules, "Control Module".In addition, the CUBE class is divided into two types: "Cubeunity", which controls the cube on the simulator, and "Cubereal", which controls real cubes.

Communication control using "Bluetooth" supports multiple platforms such as iOS and web by sending orders via a common interface.For example, Moricatron's iOS application, Urochoros, announced in November 2019, connecting a smartphone or tablet to the "toio", the cube moved around like a creature with a will.In the future, we are considering responding to Mac and Android.

The "Simulator" module consists of the mat object "MAT", the card seat object "Standard ID", and the core cube object "Cube"."CUBE" is TOIO core cube specification 2.0.Supports 0.We also implement a simulation of physical and communication delays.You can also intuitively operate with a mouse and keyboard on the simulator.The biggest advantage of using "Simulator" is that you can work while checking the cube operation on the simulator, so you do not need to build the code again every time you adjust or debug.

There are two types of control modules: "CubeHandle" for single control and "Navigator", which performs a group control.Of these, two types of methods are prepared for "CubeHandle": "CloseD-Loop" to reach the target coordinates, and "Open-Loop" for moving the specified distance or rotating to the specified angle.I am.In addition, by predicting the trajectory of movement, the continuation time of the command will be adjusted so that it does not go out of the mat.

"Navigator" implements an algorithm called "BOIDS", which can act in a group like fish groups, in addition to "Human-Like collision avoidance" that avoids other individuals with natural movements.Furthermore, by combining these two types, it is possible to reproduce the state in which each individual can move around naturally even when the cube forms a flock.In the demon -playing mode implemented in the iOS app "Urochoros" mentioned above, all the actions of chasing and escaping cubes are generated by these algorithms.

Utilization cases in a wide range of fields by testers

"TOIO SDK for Unity" has already been used by monitors such as university research groups and individual writers, and plays in a competitive game that recognize gestures and play remotely using unity network communication.Multiple utilization examples, such as how to respond to "bolt", a visual scripting environment of Unity, is introduced.

SIE and Unity Technologies Japan have held "TOIO" and the original work video contest "Robo Yaro" using "TOIO" and Unity from October 10 to November 4.Rewards will be given according to the impact of.

In addition, SwitchSCIENCE has a "TOIO" mats for developers assuming combination with "TOIO SDK for Unity".A3 size 12 pieces set, about 1 when everything is lined up.It becomes a 2m square square.Here also, multiple examples of using the core cube on the whiteboard or using a mat to create an interracable screen are introduced through the company's development campaign.

In addition, the "Simulator" module of "TOIO SDK for Unity" corresponds only to one development mat area with initial specifications, but change the coordinate system to the "Cube Handle" class of the "control module".Because of the function, it was theoretically possible simulation with 12 mats depending on the unique adjustment.

Control the robot "TOIO" freely with Unity!Introducing "TOIO SDK for Unity" from [1:20:46]

Writer: Ritsuko Kawai / Ritsuko Kawai