How to do single-character trajectory optimization?
Use Continuous-time Gaussian process motion planning via probabilistic inference.
From Theseus :
Differentiable nonlinear optimization provides a general scheme to encode inductive priors, as the objective function can be partly parameterized by neural models and partly with expert domain-specific differentiable models. The ability to compute gradients end-to-end is retained by differentiating through the optimizer which allows neural models to train on the final task loss, while also taking advantage of priors captured by the optimizer.
The goal is to find the trajectory (pose/observation and velocity/locomotion) of the robot given a start and goal pose/observation and some representation/dynamics of the environment.
(optimization initial value set to median skeleton)
Steps:
-
Considering the brush projection on paper is simply a circle (radius in a reasonable range), can we obtain an optimal trajectory that best fill each stroke’s contour using GPMP?
-
Brush model a little more complicated but more realistic, what is the result image?
-
If brush model is parameterized by neural net, what is the result?
Why doing this?
- With expert trajectories, we can do imitation learning.
- But… why not do offline RL which does not require expert data?
- Instead of performing planning for each character / stroke, why not learn a policy that can do this job for all characters?
- maybe policy training is too expensive and not worth it. Planning method will suffice.
find a path-planning using off-the-shelf optimizer project: Learning Mixed-Integer Convex Optimization Strategies for Robot Planning and Control
use mathematical optimization method (Learning Mixed-Integer Convex Optimization Strategies for Robot Planning and Control)
[theseus] and [gpmp2: Gaussian process motion planning] code are a bit complex.