ROS 2 migration pain points

pain.md

Launch

• Where the hell do I find complete docs for XML launch files?
  • https://design.ros2.org/articles/roslaunch_xml.html seems to be the canonical doc
    • but XSD schemata are machine-readable, not human-readable
    • also, it is missing lots of stuff (launch_ros.actions), so e.g. push_ros_namespace is not mentioned in this doc
  • https://docs.ros.org/en/rolling/How-To-Guides/Migrating-from-ROS1/Migrating-Launch-Files.html
    • this is a quite good source, but it isn't comprehensive
• use_sim_time hell (i.e. no support for global parameters)
  • https://discourse.ros.org/t/ros2-use-sim-time-leads-to-inconsistent-clocks/42030/2
  • <set_param> could partially remedy this (but each separately launched file has to set it)
  • Why can't ROS 2 just require a parameter_blackboard node for use_sim_time?
• $(dirname) is broken and unreliable (ros2/launch#618)
• ugly testing for nonempty XML launch args ($(eval 'bool(\'$(var var_name)\')'))
• in Python launch, launch_arguments of IncludeLaunchDescription ingest booleans as 'true'/'false', i.e. strings!
  • launch_arguments=[('use_sim_time', 'true'),]
  • and the error you get if you pass an actual boolean is super cumbersome
• cannot use parameters which are lists of dicts or other non-trivial types (ros2/ros2#1380)
  • the error you get is difficult to understand
• I haven't found any docs on writing custom launch actions (and it took me quite some time to get a little into it)
  • what's missing at minimum is a doc that would tell what are all the steps needed to register the action
    • setup.cfg, make and install as a Python package even if it's a C++ one otherwise, add the decorator...
• No XSD schema for launch_ros with all its actions, only for launch base:
  • ros2/launch#392
  • launch schema is at https://raw.githubusercontent.com/ros2/design/gh-pages/articles/specs/launch.0.1.1.xsd
  • no official example on how to use the schema in launch files
• Setting <arg value> in an <include> will also overwrite the arg in the rest of the launch file (ros2/launch#620 (comment))
  • Workaround: wrap each <include> within its own <group>.
  • It is described in the migration guide, but its meaning is not really obvious to people who want to dive into ROS 2 without prior knowledge:

    • Available in ROS 1, included content was scoped. In ROS 2, it’s not. Nest includes in group tags to scope them.

  • <param> doesn't support if="" (ros2/launch_ros#323)

rclpy

• creating Image, PointCloud2 and other messages with large arrays can be extremely slow
  • https://stackoverflow.com/a/71940633/1076564
  • workaround: assign msg._data instead of msg.data
  • alternative (better) workaround: launch Python with -O: https://docs.python.org/3/using/cmdline.html#cmdoption-O
    • how to do that? #!/usr/bin/env python -O ?

QoS

• I haven't found any complete documentation about qos_overrides parameter. There are some bits and pieces, but e.g. no docs page explains to you that deadline is in nanoseconds.

CLI tools

• ros2 topic hz is unusable because it is in Python and it deserializes all messages (https://discourse.ros.org/t/rmw-zenoh-binaries-for-rolling-jazzy-and-humble/41395/18)
  • maybe got fixed (or at least improved) by ros2/ros2cli#1005
• tab completion of ros2 topic is much worse than ROS 1. Also, the pregenerated message content is incorrect and cannot be used as is
• CLI tools silently ignore RMW_IMPLEMENTATION variable and only talk to the running daemon. So if you first launch something with one RMW and then want to run something else with another RMW, all CLI tools still silently use the first RMW. To solve that, call ros2 daemon stop.

package types

• ament_python packages contain a lot of unnecessary boilerplate: https://discourse.ros.org/t/why-is-there-no-ament-python-simple/42591
• what is a preferred package type for message-only packages? do they need to use ament_cmake? this is a quite common task, so it'd be great to have this documented
  • documented here: https://docs.ros.org/en/rolling/Tutorials/Beginner-Client-Libraries/Custom-ROS2-Interfaces.html#create-a-new-package

ros_gz

• server can be launched as a composable node, but the client can't (need to use Executable)
• topic bridges cannot be launched as composable nodes with the bridge specification passed as arg/parameter (only through YAML config)
  • there is no way how the YAML config of a bridge could be parametrized by world and robot name, which makes these configs practically useless (gazebosim/ros_gz#717)
• topic bridges do not allow easy setting of QoS profiles (but that can be overcome by setting qos_overrides manually)
  • quite important for /clock bridge!
• it is quite difficult to understand how changing a sensor frame_id works, because you have to use gz_frame_id, but that is an undocumented SDF element.
  • SDF defines frame_id, but that doesn't change the ROS frame ID
• <export>ing Gazebo resource paths in package.xml no longer works
  • This needs to be newly implemented using ament hooks and DSV files; the official project template is wrong (gazebosim/ros_gz_project_template#43) and confusing (gazebosim/ros_gz_project_template#44)

Components

• component_container is single-threaded (it even runs its own load/unload callbacks on the single thread)! Use component_container_mt or component_container_isolated.
• When component manager dies and respawns, no components are reloaded into it!!! ros2/launch_ros#361

Colcon

• no support for explicit --extend as in catkin_tools: colcon/colcon-core#393
• no integrated make job server => cannot limit num of build threads with -j1 or --parallel-workers 1. Have to use MAKEFLAGS=-j1. colcon/colcon-parallel-executor#36

micro-ROS

• agent only supports FastRTPS (notably, doesn't support Zenoh): micro-ROS/micro-ROS-Agent#243

bag files

• ros2 bag reindex can only partially fix bags whose recording was forcefully terminated: ros2/rosbag2#2003
• rqt_bag is unusable (but I'm working on it)

rviz

• doesn't handle image transport (ros2/rviz#738) !!!!!!

package types

• what is a preferred package type for message-only packages? do they need to use ament_cmake? this is a quite common task, so it'd be great to have this documented

They do have to be CMake packages (and thus use ament_cmake). It's mentioned here: https://docs.ros.org/en/rolling/Tutorials/Beginner-Client-Libraries/Custom-ROS2-Interfaces.html#create-a-new-package

Note that it is, and can only be, a CMake package, but this doesn’t restrict in which type of packages you can use your messages and services. You can create your own custom interfaces in a CMake package, and then use it in a C++ or Python node, which will be covered in the last section.

This might not be visible enough, though. Maybe we can make it a reST note.

component_container is single-threaded (it even rins its own load/unload callbacks on the single thread)! Use component_container_mt.

or component_container_isolated with single or multi threads?

Great list!

Re: launch. I've been eyeing picking up ros2/launch#392 for the launch schema. Right now the design doc is close to the real thing but it's not actually a schema used by anything and may be out of date. We absolutely want a usable schema just like for package.xml that could be automatically validated against. Including a launch_ros extension. With that, we could autogenerate human-readable documentation - because as you said it's not very ergonomic to read XSD directly.

ros2 topic hz is unusable because it is in Python and it deserializes all messages (https://discourse.ros.org/t/rmw-zenoh-binaries-for-rolling-jazzy-and-humble/41395/18)

ros2/ros2cli#1005 addressed this already 😃 i am not sure how much it helps though... 😓

The biggest one was the switch from TCPROS to ROS 2 FastDDS on Jazzy has been extremely difficult to get performing well at the same system CPU load we had in ROS 1. SHM does not work out of the box.
ros2/ros2#1289

package types

• what is a preferred package type for message-only packages? do they need to use ament_cmake? this is a quite common task, so it'd be great to have this documented
  This might not be visible enough, though. Maybe we can make it a reST note.

@christophebedard Thanks for the link. Somehow I missed it. I think the current docs are okay. The only thing that might be missing is --message and --service arguments to ros2 pkg create. These might even check that the package type is ament_cmake.

or component_container_isolated with single or multi threads?

@fujitatomoya Thanks, I've added it as another option. It might actually be a better default because it doesn't need to specify the number of threads (in single-threaded mode). However, for things like Gazebo+bridge, _mt might be better to limit the number of threads to a reasonable number.

Re: launch. I've been eyeing picking up ros2/launch#392 for the launch schema.

@emersonknapp I'm 100% for this. That would be great.

ros2/ros2cli#1005 addressed this already 😃 i am not sure how much it helps though... 😓

@fujitatomoya Great, right on time :) Let's see how it performs in reality. We need to wait for the next sync...

The biggest one was the switch from TCPROS to ROS 2 FastDDS on Jazzy

@Ryanf55 Yes, we go right away with Zenoh to minimize the DDS hassle. I wouldn't mind if Zenoh became the default because it is literally the least-hassle option. Production environments can then be switched to a DDS of their liking tuned to the specific use-case.

I'm not sure if one can make a PR for a gist, but I've compiled some additions if you would like to include any of them: https://gist.github.com/MoffKalast/8141e90841d57174571e6f4902b67378/revisions

@peci1 would you be willing to move this list into a high-level issue on https://github.com/ros2/ros2? It would be a bit easier to add items to the list. Then we can discuss there, create concrete issues, and link them back to the high-level issue.

Thanks for the comments, @MoffKalast and @christophebedard .

The main purpose of this list is to put together issues our lab has faced during the migration. Other labs can create their own lists.

If ROS 2 maintainers want to create a meta bug for migration issues, I'd certainly put some of these issues there. But I'll leave creation of such issue to the maintainers.

If ROS 2 maintainers want to create a meta bug for migration issues, I'd certainly put some of these issues there. But I'll leave creation of such issue to the maintainers.

OK, sounds good. I think I'll create it and collect things from lists such as yours and then create concrete issues in the relevant repos from there.

@christophebedard just a heads up that Martin filed an issue for

CLI tools silently ignore RMW_IMPLEMENTATION variable and only talk to the running daemon. So if you first launch something with one RMW and then want to run something else with another RMW, all CLI tools still silently use the first RMW. To solve that, call ros2 daemon stop.

here ros2/ros2cli#1009