We're hiring!

Modern USB gadget on Linux & how to integrate it with systemd (Part 2)

Andrzej Pietrasiewicz avatar

Andrzej Pietrasiewicz
March 27, 2019

Share this post:

In the previous post I introduced you to the subject of USB gadgets implemented as machines running Linux. We also talked about modern style of USB gadget creation and integrating that with systemd. In this post, we look at how to implement your very own USB function with FunctionFS and how to integrate that with systemd.

The general idea of FunctionFS

The idea is to delegate actual USB function implementation to userspace, using a filesystem interface (read()/write() etc). This way, USB out traffic (from host) is available at a file descriptor ready for reading, and USB in traffic (to host) is accepted at a file descriptor ready for writing.

What you need to know before using FunctionFS

In USB all communication is performed through so called endpoints. At the device side the endpoints are (hardware) FIFO queues associated with the UDC. There are 4 types of USB endpoints: control, bulk, iso and interrupt. Control endpoint is endpoint 0 and all USB devices shall have at least endpoint 0. This is a bi-directional communication channel between the host and the device used for enumerating and then controlling the device - remember, the USB is a host centric bus. So even if a device has some new data to be sent to the host it is the host that actually asks for data and the communication required to arrange this happens on the endpoint 0. All other types of endpoints are uni-directional. Bulk endpoints are meant for tansferring (potentially) large amounts of data on a "best effort" basis, that is, as available bandwith allows, so there are no timing guarantees for bulk data.

On the other hand bulk data is guaranteed to be transmitted error-free (perhaps using re-transmission under the hood). Iso(chronous) endpoints are meant for time-sensitive data (such as audio or video) with guaranteed bandwitdh and bounded latency. Neither data delivery nor integrity is guarenteed, though, and it is on purpose: in multimedia transmission much more harm is done by non-timely data delivery rather than by occasional non-integrity or even loss of a frame. Interrupt endpoints are for non-periodic data transfers "initiated" by the device (such as mouse movements or keystrokes) and have guaranteed latency. The "initiation" by the device is in fact queuing the data on an endpoint, and then the host polls it when it considers appropriate. In USB it's a host's world. Get used to it or quit using USB.

Why am I telling you this? You need to know that each USB function requires its own set of endpoints, which must not be used at the same time by any other function. This makes UDC's endpoints a scarce resource and imposes a limit on a number of functions which can be provided by your gadget at a time (in one configuration, but you can "overcommit" by specifying multiple configurations - you still remember, that only one of them can be active at a time?). If you want to roll your own USB function, you need to know how many and what kind of endpoints you need, and, especially if you want to compose it with other functions, whether the number of available endpoints is not exceeded. Modern UDCs usually do provide enough endpoints to compose 2-3 moderately "endpoint-hungry" functions into one configuration without any problems.

More detailed idea of FunctionFS

Now that you know about endpoints, you can learn more about FunctionFS. From the point of view of the gadget it is yet another USB function available for composing a gadget from. But it is special, because each instance of FunctionFS provides an instance of a 'functionfs' filesystem to be mounted. If you mount it you will notice there is only one entry: ep0. Its name suggests it is associated with endpoint 0. Indeed it is. However, the good news is that you don't need to implement all the endpoint 0 handling, because most of that is already taken care of for you by the composite layer. You only need to handle control requests directed at your particular USB function. ep0 is also (ab)used to specify the endpoints and USB strings. FunctionFS is not considered ready for binding until endpoint descriptors and strings are written to ep0. The gadget as a whole is not ready for binding if any of its functions is not ready. After the descriptors and strings are written, FunctionFS creates appropriate number of ep<number> files for you to use to implement the desired data streams between the host and the device.

An example function can be found in kernel sources: tools/usb/ffs-test.c The file also contains an example of how to specify USB descriptors and strings to be written to ep0 to make FunctionFS ready. Actually I'm using its modified and simplified version, which passes to the host whatever the host writes to it. We will get to its code later in this post.

And how to integrate THAT with systemd?

Compared to the previous scenario there are more steps involved before the gadget becomes operational:

  • load gadget scheme and don't enable yet (because it contains FunctionFS)
  • mount FunctionFS instance
  • write descriptors and strings
  • start the userspace daemon (e.g. ffs-test)
  • enable the gadget (now that it's ready, bind it to UDC)

It turns out that systemd already has means to automate writing the descriptors and strings to FunctionFS, and starting a userspace daemon! However, the remaining steps need to be taken care of.

The scheme

Let's call the scheme ffs_test.scheme. It describes a minimal gadget with FunctionFS, leaving some attributes at their default values:

    attrs : 
        idVendor = 0xABCD;
        idProduct = 0x1234;
    strings = ( );
    functions : 
        ffs_loopback : 
            instance = "loopback";
            type = "ffs";
    configs = ( 
            id = 1;
            name = "c";
            functions = ( 
                    name = "ffs.loopback";
                    function = "ffs_loopback";
                } );
        } );

One important thing to note is that the instance name (here: "loopback") becomes a "device" name to be used when mounting this FunctionFS instance.

The service

Let's call our example service unit usb-gadget-ffs.service. This unit is almost identical to the usb-gadget.service, except the ExecStart line:

ExecStart=/bin/gt load -o ffs_test.scheme ffs_test

The difference is that we tell gt to only (-o) load gadget's composition but not bind it, which would fail anyway because the FunctionFS instance is not ready yet.

Another difference is that the Type cannot be "simple" this time, because we need systemd not to schedule loading of dependent modules until gadget scheme is fully loaded - and, consequently, FunctionFS instance registered and made available for mounting. Units of type "simple" are considered completely run immediately, so we need to change to:


This change ensures that the dependent modules will be started only after full completion of the gt command.

Complete usb-gadget-ffs.service code:

Description=Load USB gadget scheme

ExecStart=/bin/gt load -o ffs_test.scheme ffs_test
ExecStop=/bin/gt rm -rf ffs_test


The mount

We can use a mount unit to automatically mount our FunctionFS instance. We choose to mount it at /run/ffs_test, so the mount unit must be named run-ffs_test.mount:

Description=Mount FunctionFS instance

# "device" name (FunctionFS instance name)


and then use systemctl enable run-ffs_test.mount

The above two units are enough to load our gadget composition into memory at UDC's appearance and mount its accompanied FunctionFS instance.

Descriptors, strings and the userspace daemon

systemd supports socket units capable of listening to usb traffic directed at FunctionFS. Such a socket unit must be pointed at an already mounted FunctionFS instance. The socket unit contains ListenUSBFunction entry in its [Socket] section to specify exactly that. It also contains a Service entry pointing to the service unit associated with this unit and such a socket unit must have its associated service unit.

When the socket unit is started, it starts its associated service unit, which contains USBFunctionDescriptors and USBFunctionStrings entries. The two specify locations of files containing binary blobs representing USB descriptors and strings, respectively. The service unit's job is then to write those to ep0 of the corresponding FunctionFS instance. And here the systemd's lazy daemon start comes into play: the descriptors and strings are already passed to FunctionFS, so it becomes ready _but_ the daemon is _not_ started until some traffic directed at this FunctionFS instance happens on USB. And it is exactly the job of the socket unit to make it work. The daemon binary is specified with the usual ExecStart entry in the service unit.

The socket unit

Let's call it ffs.socket:

Description=USB function fs socket

# we will get to ExecStartPost later
ExecStartPost=/bin/gt enable ffs_test


And then systemctl enable functionfs.socket.

The service unit accompanying the socket unit

We call it functionfs-daemon.service as per the Service entry above.


A question you likely ask yourself is where to get the .bin files from? include/uapi/linux/usb/functionfs.h in the kernel sources provides the format description (line 89 in v5.0-rc6). It is up to you how you create these blobs. An example of how to create the descriptors (and strings) in a C program can be found in tools/usb/ffs-test.c. Please note that you are supposed to use usb_functionfs_descs_head_v2, because the other format is now deprecated. You can modify the program so that it doesn't do anything except writing the descriptors/strings to standard output and then capture the result in a file.

Here is a hex dump of descriptors and strings blobs for a high-speed-only example funtion:

# hd descriptors-ffs-test.bin 
00000000  03 00 00 00 27 00 00 00  02 00 00 00 03 00 00 00  |....'...........|
00000010  09 04 00 00 02 ff 00 00  01 07 05 81 02 00 02 00  |................|
00000020  07 05 02 02 00 02 01                              |.......|

# hd strings-ffs-test.bin 
00000000  02 00 00 00 1d 00 00 00  01 00 00 00 01 00 00 00  |................|
00000010  09 04 55 53 42 20 46 69  6c 74 65 72 00           |..USB Filter.|

Please note that the descriptors are created only in the unmodified version of ffs-test.c. As promised, we will get to the modified version later in this post.

Enabling the gadget

If the functionfs.socket did not contain the ExecStartPost entry, then at this point we would have a gadget ready to be bound, but not actually bound. The ExecStartPost contains a command which executes our gadget binding _after_ the service unit is started, which is the last missing piece in this puzzle. Your gadget is now composed and activated with systemd.

Can I use existing daemon code to integrate with systemd?

No, you can't. However, usually patching userspace code to be able to be passed open file descriptors by systemd is a trivial task. And to make it even easier for you to play with FunctionFS and systemd I have modified the ffs-test.c program for using with systemd (it purposedly contains some simplifications in order for you to be able to follow the code easily, so you can't take it as a full-fledged implementation):


look for tools/usb/ffs-test.c on the ffs-systemd branch.

Giving it a try

In order to test the above mentioned modified function you need the following:

  • Scheme of the example gagdet from this post
  • All systemd units from this post
  • ffs-test binary compiled from the modified sources
  • FunctionFS endpoint descriptors and strings binary blobs
  • usbserial module at host with generic serial support

Set up your gadget and connect it to the host, it should enumerate correctly. The idea is to bind generic usb serial driver at the host to our device. This results in automatic creation of ttyUSB<X> at the host side. And then whatever you write to the ttyUSB<X> you can read back from it. That's what the modified ffs-test.c does - a loopback function.

# at the host
echo 0xabcd 0x1234 > /sys/bus/usb-serial/drivers/generic/new_id

usbserial_generic 3- The "generic" usb-serial driver is only for testing and one-off prototypes.
usbserial_generic 3- Tell linux-usb@vger.kernel.org to add your device to a proper driver.
usbserial_generic 3- generic converter detected
usb 3- generic converter now attached to ttyUSB1

while true; do dd if=/dev/ttyUSB1 bs=1 iflag=nocache status=none 2>/dev/null; done

And still at the host, at some other console:

man -P cat bash > /dev/ttyUSB1

The contents of bash man page should appear at the console where the "while" loop runs. You can further modify ffs-test.c to e.g. capitalize all the letters, or use some encryption algorithm to cipher your text, effectively creating a hardware crypto device on USB!

What's next?

In the next installment I will write about using dummy_hcd. And in yet another one we will add some systemd templatization to make the above solutions more generic.

Comments (16)

  1. J. Rossbach:
    Aug 19, 2019 at 03:11 PM

    If I have multiple interface descriptors (with different interface numbers, i.e. not alternatives to each other), can I use the same FunctionFS instance to handle them all or do I need to mount one FFS instance per interface descriptor?

    Reply to this comment

    Reply to this comment

    1. Andrzej Pietrasiewicz:
      Aug 26, 2019 at 09:06 PM

      Yes, you can use single FunctionFS instance.

      Reply to this comment

      Reply to this comment

      1. J. Rossbach:
        Aug 29, 2019 at 08:44 AM

        Ok, but in this case, how can I distinguish EP0 events for the different interfaces? E.g. let's assume I have two interfaces each having two alternative settings (i.e. four interfaces altogether). If the alternative setting is changed for one of the interfaces, how can I determine, for which of the two interfaces this happened and which alternative has been activated? If I would use a dedicated FFS instance for each interface on the other hand, I would have dedicated EP0s and could distinguish thereby (however I still do not know how to obtain which alternative was activated).

        Reply to this comment

        Reply to this comment

  2. Sam Andre:
    Jul 13, 2020 at 03:55 PM

    Hello and thank you for this interesting article !
    i am trying to compile ffs-test and i get the following errors:
    sudo gcc ffs-test.c
    /usr/bin/ld: /tmp/ccJHXqOP.o: in function `start_thread_helper':
    ffs-test.c:(.text+0x89c): undefined reference to `__pthread_register_cancel'
    /usr/bin/ld: ffs-test.c:(.text+0x8a0): undefined reference to `pthread_testcancel'
    /usr/bin/ld: ffs-test.c:(.text+0x9b4): undefined reference to `__pthread_unregister_cancel'
    /usr/bin/ld: /tmp/ccJHXqOP.o: in function `start_thread':
    ffs-test.c:(.text+0xa50): undefined reference to `pthread_create'
    /usr/bin/ld: /tmp/ccJHXqOP.o: in function `join_thread':
    ffs-test.c:(.text+0xab8): undefined reference to `pthread_join'
    collect2: error: ld returned 1 exit status

    Reply to this comment

    Reply to this comment

    1. SamAndre:
      Jul 14, 2020 at 11:35 AM

      OK i mixed up with the deprecated file.
      Now i used the new ffs-test.c file
      the following steps i have done
      is something wrong with it or missunderstood?
      -> ffs_test.scheme
      -> usb-gadget-ffs.service
      -> run-ffs_test.mount ( systemctl enable run-ffs_test.mount )
      -> ffs.socket (systemctl enable ffs.socket )
      -> functionfs-daemon.service
      used makefile from system-branch to compile ffs-test.c
      moved binary ffs-test to /bin/
      no response when the pi is connected to the host

      Reply to this comment

      Reply to this comment

    2. Sam Andre:
      Jul 14, 2020 at 12:47 PM

      OK i mixed up with the deprecated file.
      Now i used the new ffs-test.c file
      the following steps i have done
      is something wrong with it or missunderstood?
      -> ffs_test.scheme
      -> usb-gadget-ffs.service
      -> run-ffs_test.mount ( systemctl enable run-ffs_test.mount )
      -> ffs.socket (systemctl enable ffs.socket )
      -> functionfs-daemon.service
      used makefile from system-branch to compile ffs-test.c
      moved binary ffs-test to /bin/
      no response when the pi is connected to the host

      Reply to this comment

      Reply to this comment

      1. Andrzej Pietrasiewicz:
        Jul 14, 2020 at 05:02 PM

        Did you provide descriptor and string blobs?

        Reply to this comment

        Reply to this comment

  3. Sam Andre:
    Jul 15, 2020 at 04:19 PM

    invoking ffs-test results in :
    ffs-test: Cannot inherit descriptors from systemd: (-0) Success

    i hope i have not missunderstood the tutorial.
    compiled ffs-test should create the descriptor and string blobs right ?

    Reply to this comment

    Reply to this comment

    1. Andrzej Pietrasiewicz:
      Jul 16, 2020 at 03:54 PM

      "Please note that the descriptors are created only in the unmodified version of ffs-test.c." so to create the descriptors you need to build the unmodified version, too. Or use a hex editor to hand-craft them.

      The "Cannot inherit descriptors from systemd" message likely means that either something is mis-configured or you use systemd version which does not support FunctionFS.

      Reply to this comment

      Reply to this comment

      1. SamAndre:
        Jul 26, 2020 at 09:37 PM

        Thanks for your answer and help! Yes there is a deeper problem/misconfiguration.
        To find the problem i maked a step back and tried the ecm gadget from page 1 and i found the following problem :
        I can successfully load the gadget via console with "gt load ecm.scheme ecm" ,
        but when i try the systemd solution, "usb-gadget.service" reports the following error:

        "usb-gadget.service - Load USB gadget scheme for ecm
        Loaded: loaded (/etc/systemd/system/usb-gadget.service; disabled; vendor preset: enabled)
        Active: failed (Result: exit-code) since Sun 2020-07-26 19:35:16 BST; 1h 51min ago
        Main PID: 259 (code=exited, status=255/EXCEPTION)

        Jul 26 19:35:13 raspberrypi systemd[1]: Started Load USB gadget scheme for ecm.
        Jul 26 19:35:13 raspberrypi gt[259]: Could not find matching gadget file.
        Jul 26 19:35:16 raspberrypi systemd[1]: usb-gadget.service: Main process exited, code=exited, status=255/EXCEPTION
        Jul 26 19:35:16 raspberrypi systemd[1]: usb-gadget.service: Failed with result 'exit-code'.
        Dont know why the service cant find the scheme file , they are both in the same folder !
        I did a fresh installation of Raspian Buster and got the same result again.

        Reply to this comment

        Reply to this comment

          1. Sam Andre:
            Sep 16, 2020 at 11:19 PM

            Thanks for your answer !
            Meanwhile i got it working partly and i detected some problems.
            I created the string and descriptor blobs exactly as described but i have a problem with the second Endpoint-Descriptor [07 05 02 02 00 02 01] .
            The result of "USB Descriptor Dumper" shows the second Endpoint-Descriptor is configured as "0x01 ( OUT endpoint 1 )" instead of "0x02 ( OUT endpoint 2)".
            I have no Idea why its being ignored. The first Endpoint-Descriptor is configured correct.

            Second Question
            How can i set the Device-Descriptor with Data from the String Descriptor-Table in this case?
            things like:
            iManufacturer = 0x01 ;
            manufacturer = 0x01 ;
            on attrs-area in the scheme-file not working!

            Reply to this comment

            Reply to this comment

            1. Andrzej Pietrasiewicz:
              Sep 17, 2020 at 03:26 PM

              Did you properly set bNumEndpoints? If it equals, say, "1" then chances are the other descriptor is never taken into account.

              As for manufacturer string, try this in the gadget scheme:

              strings = (
              lang = 0x409;
              manufacturer = "Collabora";
              product = "MTP Gadget";

              Reply to this comment

              Reply to this comment

  4. stuomas:
    Mar 26, 2021 at 10:50 AM

    Have you tried this with a dual role USB port? I use the same systemd files like in your Linux Piter 2019 youtube video, and I'm suffering from synchronous external aborts/kernel panics often when detaching a USB stick, and extcon driver changes role from host to device.

    My gadget is RNDIS+MTP.

    If I disable the gadget first (gt disable ffs_mtp) and then disconnect and role changes, everything is fine. Then I have to gt enable ffs_mtp again when I want to use it. I have automatized this with udev rules, but it feels like circumventing the actual problem.

    I have no idea where the problem is, libusbgx, gt (using your fork), cmtp-responder, these systemd files, USB driver, hardware? So maybe not related to this post, just wondering if you have had any similar experiences.

    Reply to this comment

    Reply to this comment

  5. Marek:
    Sep 10, 2021 at 07:31 AM

    Thanks a lot for valuable info. I have integrated this way my custom ffs based gadget and everything woks smoothly (host is windows). I upgraded kernel from 4.12 to 5.4 and not on windows I'm seeing that device is not recognized properly. I didn't change anything in regard of gadget setup. Any ideas what else could I check? When run script which setup gadget in 5.4 it works fine. Thanks.

    Reply to this comment

    Reply to this comment

    1. Marek:
      Sep 10, 2021 at 04:17 PM

      Found it. Seems that missing serial number in gt scheme like: serialnumber = "85246"; was missing (maybe was pre-populated on 4.12 but not on 5.4). When added this line everything works fine. Sorry for noise but maybe it helps someone ;)

      Reply to this comment

      Reply to this comment

Add a Comment

Allowed tags: <b><i><br>Add a new comment:

Search the newsroom

Latest Blog Posts

Visual-inertial tracking for Monado


Monado now has initial support for 6DoF ("inside-out") tracking for devices with cameras and an IMU! Three free and open source SLAM/VIO…

Spotlight on Meson's full-featured developer environment


When developing an application or a library, it is very common to want to run it without installing it, or to install it into a custom prefix…

How to write a Vulkan driver in 2022


An incredible amount has changed in Mesa and in the Vulkan ecosystems since we wrote the first Vulkan driver in Mesa for Intel hardware…

Improving the reliability of file system monitoring tools


Every file system used in production has tools to try to recover from system crashes. To provide a better infrastructure for those tools,…

PipeWire: A year in review & a look ahead


The PipeWire project made major strides over the past few years, bringing shiny new features, and paving the way for new possibilities in…

Landing a new syscall, part 1: What is futex?


Over the past 18 months, we have been on a roller-coaster ride developing futex2, a new set of system calls. As part of this effort, the…

Open Since 2005 logo

We use cookies on this website to ensure that you get the best experience. By continuing to use this website you are consenting to the use of these cookies. To find out more please follow this link.

Collabora Ltd © 2005-2022. All rights reserved. Privacy Notice. Sitemap.