dev-resources.site

for different kinds of informations.

How to use Flatbuffers in a C++ project with Conan?

Published at

2/2/2024

How flatbuffer works

A fbs schema file specifies the structures of the data to be serialised
flatc compiler generates a header file based on that fbs schema
This header file, along with flatbuffers library will be used in your project to serialise the data

First step: Creating a very simple flatbuffer schema file

IMPORTANT: Note the root_type Frame at the end. If you don't add a root type, flatc does not generate the deserialisation functions

namespace Camera;

table Frame {
    width: uint32;
    height: uint32;
    image_data: [uint8];
}

root_type Frame;

Second step: Adding flatbuffers to conan

In the root of your project, create a file conanfile.py in the root of your project with this content:

from conan import ConanFile
from conan.tools.cmake import cmake_layout, CMakeToolchain

class ConanApplication(ConanFile):
    package_type = "application"
    settings = "os", "compiler", "build_type", "arch"
    generators = "CMakeDeps"

    def layout(self):
        cmake_layout(self)

    def generate(self):
        tc = CMakeToolchain(self)
        tc.user_presets_path = False
        tc.generate()

    def requirements(self):
        requirements = self.conan_data.get('requirements', [])
        for requirement in requirements:
            self.requires(requirement)

Now, create a file conandata.yml in the same location as conanfile.py with this content (or add the last line to your existing conandata.yml)

requirements:
  - "flatbuffers/23.5.26"

Run conan install to get the dependencies and add it to the build directory

cd ~/my-project
conan install . --output-folder=build --build=missing --settings=build_type=Debug

Third step: Changing CMakeLists.txt to parse fbs files

Before, don't forget to see the result of the conan install. It will tell you the extra parameters you need to add when running cmake generate step.

Example:

cd ~/my-project
cmake -G "Ninja" -S . -B build -DCMAKE_TOOLCHAIN_FILE=/home/thiago/src/my-project/build/build/Debug/generators/conan_toolchain.cmake -DCMAKE_POLICY_DEFAULT_CMP0091=NEW -DCMAKE_BUILD_TYPE=Debug

In your CMakeLists.txt file:

This will run flatc to compile the fbs files and generate the header files

# Use find_package to create the cmake variables of flatbuffers
find_package(flatbuffers CONFIG REQUIRED)

# flatbuffers scheme to be compiled by flac binary
set(FB_SCHEMA "${CAMNODE_FBS_DIR}/camera-frames.fbs")
# Location of the generated files. I like to use CMAKE_CURRENT_BINARY_DIR
# So it will not be in the source code tree.
set(FB_OUTDIR "${CMAKE_CURRENT_BINARY_DIR}/fbs/")

# This is an utilitary function that is available in flatbuffer cmake integration
# fbschemas is a new target that will be created with the generated file
build_flatbuffers("${FB_SCHEMA}" "" fbschemas "" "${FB_OUTDIR}" "" "")

And this will add them as a dependency of your project

# Now we create an interface library FlatbuffersTarget to add flatbuffers includes
add_library(FlatbuffersTarget INTERFACE)
target_include_directories(FlatbuffersTarget INTERFACE ${flatbuffers_INCLUDE_DIR})
add_dependencies(FlatbuffersTarget fbschemas)

# And add FB_OUTDIR to the include directories of your project's target
target_include_directories(my-project
        PUBLIC
        # ...
        ${FB_OUTDIR} # For flatbuffers generated files
)

# As well as to the project libraries
target_link_libraries(my-project PRIVATE
        # ...
        flatbuffers::flatbuffers
        FlatbuffersTarget
)

Forth step: Serialising your struct

In the application saving or sending the flatbuffer data

FlatBufferBuilder is the object that keeps the state of the serialised data. All the other builders will add data to it and mode its offset

// Include the generated header in your file
#include "camera-frames_generated.h"
// ...

// Creates a FlatBufferBuilder instance
flatbuffers::FlatBufferBuilder fbb;

// Option 1: Use a builder

// Note that I am creating a vector BEFORE creating the builder.
auto fb_vector = fbb.CreateVector(frame->image_data);
// Creates a builder using fbb instance
Camera::FrameBuilder builder(fbb);
builder.add_width(frame->width);
builder.add_height(frame->height);
builder.add_image_data(fb_vector);
// Finishes the object increasing the offset
auto fb_msg = builder.Finish();

And now, let's get the serialised data.

IMPORTANT: fbb.Finish must be called before fbb.GetBufferPointer()

// Tells the FlatBufferBuilder instance that we finished serialising our data and
// we are ready to read the buffer
fbb.Finish(fb_msg);

// Returns a pointer to the serialised data
uint8_t* buffer = fbb.GetBufferPointer();
// And the size of the array
size_t size = fbb.GetSize();

// And how do whatever you want this this data.

If you are reusing fbb, don't forget to clear it's internal state

fbb.Clear();

Fifth step: Deserialising your struct

Last but not least, it does not make sense to serialise data if this is never going to be deserialised.
Thankfully it's much simpler

std::vector<uint8_t> buffer = recv_my_data(); 
auto frame = Camera::GetImageFrame(buffer.data());
// And you can use as it was a regular C++ class
logger->info("Frame received that I am ignoring for some reason: Dimensions=({}, {}), Size={}",
    frame->width(), frame->height(), dec_frame->image_data()->size());