Browse Author

luis

JPEG encoding on the Odroid C1/C1+

The Odroid C1+ is one of the most cost-effective yet still reasonably powerful boards out there that I know, featuring the Amlogic 32-bit S805 ARM processor. Although the Odroid C1+ is meant to be as open as possible, the documentation is quite short (if not inexistent) when it comes to video hardware encoding or decoding.

odroidc1

On that line, one of the advertised features of the processor is the ability to do JPEG hardware encoding. Unfortunately, you don’t get much more than the advertisement itself really. I think there is some Android library around that helps you get it done, but anyway, here is how you do it regardless of your Linux flavour.

Just a quick note: I say this is how you do it, but actually it is not; I only made this up by looking into the driver code, kind of figuring out how it works, and doing a fair amount of trial and error, so no warranties are given! You can find the driver in the file drivers/amlogic/amports/jpegenc.c of the Amlogic kernel tree.

Getting the board ready for action

So, let’s get the board to do some encoding. First thing you need to do, is to make sure you have JPEG encoding enabled on your kernel. As of this writing, the latest pre-built kernel has the module built-in (kernel build option AM_JPEG_ENCODER), so you probably don’t need to rebuild your kernel.

https://github.com/hardkernel/linux/tree/odroidc-3.10.y

Unfortunately, the encoding device itself is not enabled. In order to enable it, you need to add the following to your device tree:

jpegenc {
    compatible = "amlogic,jpegenc";
    dev_name = "jpegenc.0";
    status = "okay";
    reserve-memory = <0x01800000>;  //24M
    reserve-iomap = "true";
};

If you don’t know what I am talking about, but still want to try hardware encoding, despair not. Here is a jpegenc-enabled compiled version of the C1+ device tree, tested on kernels 3.10.80-121 and 3.10.80-131 (and with good chances to work on any odroidc-3.10.y kernel). Just download it, copy it to the boot partition (normally mounted under /media/boot/) with the name ‘meson8b_odroidc.dtb’, and reboot the board (remember to make a backup copy of the old ‘meson8b_odroidc.dtb’ file).

Licenses are important! Like the rest of the kernel, device trees are released under the GPLv2 license, so here is the corresponding source code of the compiled device tree. (If you don’t think this is important, keep in mind that quite often it is thanks to copyleft licenses that we users can hack with things like the jpegenc module!).

If everything has gone well, you should be able to see the file /dev/jpegenc in your file tree. That’s the file we will be using to write our raw image data, and read our encoded jpeg image.

Module memory allocation

Let’s stop for a moment and have a look at how the driver allocates and uses memory. This is not needed to start doing your own encoding, so feel free to jump to the next section.

The jpegenc module requests a single big buffer and does all the work on it. It uses contiguous memory allocation (CMA), which means the memory is physically continuous on memory. The module splits the buffer into two parts, the first for raw data input, and the other for encoded JPEG output. You can see the buffer definitions at the top of the jpegenc.c file. In our case it looks like this:

    },{
        .lev_id = JPEGENC_BUFFER_LEVEL_5M,
        .max_width = 2624,
        .max_height = 2624,
        .min_buffsize = 0x1800000,
        .input = {
            .buf_start = 0,
            .buf_size = 0x13B3000,
        },
        .bitstream = {
            .buf_start = 0x1400000,
            .buf_size = 0x400000,
        }
    },{

I say ‘our case’, because there are other buffer definitions depending on how much memory the module requests. The buffer size here is 24 MB (0x1800000), defined by min_buffsize. It also defines the maximum width and height for the image, and the input and output sub-buffers. Note that output is referred to as bitstream

If you need to work with bigger images, you should change the reserve-memory attribute on the device tree, see previous section. With the right setting, you can go as high as 8192×8192. However, though I didn’t test it myself, you may need to recompile the module because there seems to be a bug on the bitstream (output) buffer size definition for sizes bigger than ours:

    },{
        .lev_id = JPEGENC_BUFFER_LEVEL_HD,
        .max_width = 8192,
        .max_height = 8192,
        .min_buffsize = 0xc400000,
        .input = {
            .buf_start = 0,
            .buf_size = 0xc000000,
        },
        .bitstream = {
            .buf_start = 0xc000000,
            .buf_size = 0x4000000,		/* Wrong! Drop a zero: 0x400000 */
        }
    }

Anyway, back to our discussion: In order to work with the jpegenc module, we will be mapping this big buffer, and use it to write our raw image and read the encoded output image. We already know the offset for the output (bitstream) buffer (.buf_start = 0x1400000,), but userspace programs should get this information by querying the module with the right ioctl() call, as we’ll see below.

Let’s code!

To do our encoding, we start by opening the /dev/jpegenc file:

#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <sys/ioctl.h>
#include <sys/mman.h>
#include <errno.h>
#include <stdio.h>
#include <string.h>

#include "jpegenc.h"

int main()
{
	/* Open the jpeg encoder file. It may require root permissions */
	int fd = open("/dev/jpegenc", O_RDWR);
	if(fd == -1)
	{
		perror("Error on jpegenc open");
		return 1;
	}

The file jpegenc.h must be included here. It can be found in the Amlogic kernel tree, or here. I copy it to my local folder, since it doesn’t seem to be among the installed linux headers.

Once the jpegenc file is opened, we can query the module buffer info (as discussed in the previous section):

	/* Query and print buffer info */
	unsigned int addr_info[5];
	ioctl(fd, JPEGENC_IOC_GET_BUFFINFO, addr_info);
	printf("Total buffer size: 0x%08X\n"
		"Input buffer -> start: 0x%08X, size: 0x%08X\n"
		"Bitstream (output) buffer -> start: 0x%08X, size: 0x%08X\n",
		addr_info[0], addr_info[1], addr_info[2],
		addr_info[3], addr_info[4]);

I just pass an array of unsigned int to the ioctl() call, but it may be better to define a proper struct. The printf() call prints what every number means.

The following ioctl() tells the module to initialize the encoder (things like power-on, canvas initialization, microcode loading and interrupt request).

	ioctl(fd, JPEGENC_IOC_CONFIG_INIT, 0);

Next, we tell the driver what image size we will be using:

	/* Set image width and height */
	int width, height;
	width = 1280;
	height = 720;
	ioctl(fd, JPEGENC_IOC_SET_ENCODER_WIDTH, &width);
	ioctl(fd, JPEGENC_IOC_SET_ENCODER_HEIGHT, &height);

With our current device tree setting, we could go as high as 2624×2624, though I haven’t tried it myself.
It is now a good time to map the module buffer to our memory space:

	/* Map the jpegenc module "big buffer" to userspace */
	unsigned char *data;
	data = mmap((caddr_t)0, addr_info[0], PROT_READ | PROT_WRITE,
			MAP_SHARED, fd, 0);
	if (data == (void*)-1)
	{
		perror("Error on mmap");
		return 1;
	}

At this point, the module is expecting us to put our raw frame at data + offset, where offset is whatever value we have at addr_info[1] (see the JPEGENC_IOC_GET_BUFFINFO ioctl() call above). However, this value (addr_info[1]) happens to be zero for any device tree configuration, at least with kernels up to 3.10.80-131, so really you could very much ignore it.

	/* Load the raw image to the mapped buffer */
	FILE *fin = fopen("image.raw", "r");
	int ret = fread(data + addr_info[1], 1, 1280*720*3, fin);
	if (ret != 1280*720*3)
	{
		printf("Could not read full frame size\n");
		return 1;
	}

Here, I load my raw data from a file, but obviously it could be loaded from anywhere.

Even though my raw image is encoded as RGB888, you may well use any of the following color configurations, as shown in jpegenc.h (I’m afraid I didn’t test any other than RGB888 so far, comments are welcome if you successfully use another):

typedef enum{
    FMT_YUV422_SINGLE = 0,
    FMT_YUV444_SINGLE,
    FMT_NV21,
    FMT_NV12,
    FMT_YUV420,    
    FMT_YUV444_PLANE,
    FMT_RGB888,
    FMT_RGB888_PLANE,
    FMT_RGB565,
    FMT_RGBA8888,
    MAX_FRAME_FMT 
}jpegenc_frame_fmt;

Once we passed our raw data to the jpegenc driver, we can ask it to start encoding. To do so, we use an array (or a properly defined struct) of 7 unsigned int. The following code shows the meaning of them.

	/* Start encoding now */
	unsigned int encode_cmd[7];
	encode_cmd[0] = LOCAL_BUFF;		//memory type: LOCAL_BUFF probably,
						//PHYSICAL_BUFF or  CANVAS_BUFF don't seem to 
						//be possible from userspace.
	encode_cmd[1] = FMT_RGB888;		//Input format
	encode_cmd[2] = FMT_YUV422_SINGLE;	//Output format
	encode_cmd[3] = 0;			//Ignored
	encode_cmd[4] = 0;			//offset: for LOCAL_BUFF, set to 0
	encode_cmd[5] = width*height*3;	//Raw image size in bytes
	encode_cmd[6] = 1;			//need_flush: set to 1 for LOCAL_BUFF, maybe not needed.

	ioctl(fd, JPEGENC_IOC_NEW_CMD, encode_cmd);

As you see, there is a lot of guessing from myself involved here. In the driver (jpegenc.c), these values are passed to a function called set_jpeg_input_format(), refer to it if you are interested on what these values do.

If you are trying to encode a non RBG888 frame, you will need to change the value passed in encode_cmd[1] with one of the values from the enum above.

As far as I know, all the ioctl() calls we have made so far returned once the action was complete. This is not the case here: JPEGENC_IOC_NEW_CMD is a “non-blocking” command. So we will have to poll until completion, using the JPEGENC_IOC_GET_STAGE ioctl():

	/* Loop wait for the encoder to finish */
	unsigned int stage = ENCODER_IDLE;
	while (stage != ENCODER_DONE)
	{
		ioctl(fd, JPEGENC_IOC_GET_STAGE, &stage);
		printf("Stage is %d\n", stage);
		usleep(1000);
	}
	printf("Job done!\n");

Once the driver returns ENCODER_DONE, we are ready to grab our compressed JPEG frame! Where should we get it from? Back to the first ioctl() at the top of the code, the value of addr_info[3] was the offset for the bitstream (output) part of the big buffer we mapped with mmap (which we just called data here).

So, we can read our compressed frame at data + addr_info[3] but, where does it end? Conveniently, we can query the size of the output data:

	/* Query the output size */
	unsigned int output_size = 0;
	ioctl(fd, JPEGENC_IOC_GET_OUTPUT_SIZE, &output_size);
	printf("Output size is %d bytes\n", output_size);

And that’s it! Here is an example of saving the encoded frame into a file:

	/* Write the image to file */
	FILE *fout = fopen("./image.jpeg", "w+");
	int written = fwrite(data + addr_info[3], 1, output_size, fout);
	if (written != output_size)
	{
		printf("Could not write the full output file\n");
		return 1;
	}

You can get the full example code here. Remember to put jpegenc.h in the same folder. You can compile the example from the Odroid itself with:

$ gcc main.c -o myencoder

Other stuff

Although this code needs to be run as root, you can easily change that by running:

$ sudo chmod 666 /dev/jpegenc

That should allow any user doing some encoding.

On my board, the code loops 6 times on the wait loop before finishing, which means it takes roughly 6ms to encode a 720p image. If things scale up nicely, that means it possibly could do 100fps+ MJPEG encoding, with very little use of the CPU! (And, hopefully, not heating up too much!)

Finally, there are a bunch of ioctl() I haven’t discussed here, since this is only a basic example. In fact, I haven’t even tried them myself, but it seems to be possible to set different qualities, or tell the encoder to use your own quantization table (I’ll leave that to you, intrepid reader).

Please, leave me any comments for any corrections you may spot, or any progress you do on your own with the Odroid C1+ video hardware!

Happy hacking!

4 Comments

A UDP client example with GLib / GIO

Here is a simple UDP client made with GIO (GLib). GIO is a great library to use once you get up to speed on it, but sometimes I struggle to find good examples for even the most basic things, and end up building things up from scratch using the library documentation, which sometimes can be a bit tricky. So in case it is useful for someone, here it goes!

Please post any corrections.


#include <unistd.h>		/* For sleep() */
#include <string.h>		/* For strlen() */
#include <stdio.h>
#include <gio/gio.h>

int main()
{
	char *host = "127.0.0.1";
	short port = 23456;
	
	GSocket *sock;
	GSocketAddress *sock_address;
	GError *err = NULL;
	
	sock = g_socket_new(G_SOCKET_FAMILY_IPV4,
							G_SOCKET_TYPE_DATAGRAM,
							G_SOCKET_PROTOCOL_UDP,
							&err);
	
	if (err != NULL)
	{
		printf ("%s\n", err->message);
		return 1;
	}
	
	sock_address = g_inet_socket_address_new_from_string(host, port);

	while(1)
	{
		char *some_text = "Some text\n";
		g_socket_send_to(sock, sock_address, some_text, 
												strlen(some_text), NULL, &err);
		
		if (err != NULL)
		{
			printf ("%s\n", err->message);
			return 1;
		}
		
		sleep(1);
	}

	return 0;
}
Comments Off on A UDP client example with GLib / GIO

Using spidev on the Wandboard

No matter how well documented a particular device is, I usually stumble trough a considerable number of trial-and-error iterations whenever I try to do some development on it. That’s why, for the initial stages of development, I normally try to use some computer on board platform running an operating system, rather than creating my own microcontroller code from scratch.

Unfortunately, there’s a good chance that you’ll find issues no matter what path you follow, and trying to use the SPI bus on my Wandboard was no exception.

Just to make it clear, I think the Wandboard is a great development platform, very stable and well documented. But I faced a couple of problems: first, the SD image I downloaded had no spidev support out of the box, and second I messed with it and then read the manual, in that order. However, the effort spent on getting spidev going pays off very quickly, giving you the ability to talk to the SPI bus from userspace.

Note: This method uses device tree, so it won’t work with kernel versions < 3.10

Loading the spidev module

The first thing you want to do is try to load spidev in case it is already on your installation. To do so, make sure the module is loaded:

$ sudo modprobe spidev

At this point I got a “not found” error. So time to get the module myself. Since I didn’t have any prebuilt kernel in my system, I downloaded the kernel source (https://github.com/wandboard-org/linux selecting the right branch) and compiled a new kernel for the wandboard. The reason why I opted to upgrade the kernel instead of just adding the module is that the preinstalled kernel had a release version that didn’t completely match any of the available ones at github.

Building a kernel can be a whole topic on itself, but I’ll try to give a quick overview of what I did.

Once the kernel is downloaded in the Wandboard, get the kernel configuration ready:

$ make ARCH=arm wandboard_defconfig

Now we need to enable spidev as a module in the configuration. There are many ways to do this, probably the easiest is just to edit .config manually; however, I like using ncurses config:

$ make menuconfig

If you get an error, chances are you are missing libncurses headers. Run

$ sudo apt-get install libncurses5-dev

(or similar) to fix it.
The linux installation on my Wandboard was using zImage for booting. So I ran:

$ make ARCH=arm zImage
$ make ARCH=arm modules
$ sudo make modules_install

(You probably don’t need the “ARCH=arm” bit, especially since we are not cross compiling)

Make sure you backup the old zImage before replacing it with the new one. If things get bad and the new zImage fails to boot, you can always get the SD card off the Wandboard and restore the old zImage using a card reader on your desktop machine.

Finally, you should be able to modprobe spidev on your system. Have a look at /dev/, and see if you can find any file called spidev1.0 or similar. If you don’t see any, as in my case, it means you are not done yet… The following section may still be useful even if you have the file, to make sure your pin configuration is fine.

The device tree

What happens is, you have the spidev module with all the SPI stuff in (we didn’t talk about the IMX SPI driver, but that comes built-in by default), but spidev doesn’t know which wandboard pins to use. That’s because we haven’t defined them in the device tree yet.

In case you don’t know what I am talking about: in the olden days, there used to be lots of C files in the kernel describing the different computer boards out there. Those files have been replaced by device tree files. From devicetree.org:

The Device Tree is a data structure for describing hardware. Rather than hard coding every detail of a device into an operating system, many aspect of the hardware can be described in a data structure that is passed to the operating system at boot time.

You can find the device tree files for ARM boards in arch/arm/boot/dts/ in the kernel. The files we are going to look at are imx6qdl.dtsi and imx6qdl-wandobard.dtsi (This works for the Wandboard dual/quad, I have no Wandboard solo with me, but the process should be similar). Most of this I got from the Wandboard forums.

Of the 5 spi modules (actually called ecspi) in the Wandboard, we are going to use ecspi1, which is the one routed to the expansion connector. So you need an ecspi1 section in your imx6qdl-wandboard.dtsi file:

&ecspi1 {
	fsl,spi-num-chipselects = <2>;
	cs-gpios = <&gpio2 30 0>, <&gpio4 10 0>;
	pinctrl-names = "default";
	pinctrl-0 = <&pinctrl_ecspi1_1>;
	status = "okay";

	spidev@0x00 {
		compatible = "spidev";
		spi-max-frequency = <16000000>;
		reg = <0>;
	};
	
	spidev@0x01 {
		compatible = "spidev";
		spi-max-frequency = <16000000>;
		reg = <1>;
	};
};

I am not much of an expert of device trees myself so don’t take my word for it, but what we are doing here is basically telling the kernel to probe the spidev module twice. To do so, we use the compatible property, in both spidev sections.

The reg property tells the spidev module which chip select to use; so we have an spidev module for chip select 0, and one for chip select 1. It is just an index though; the real chip select lines are defined in the cs-gpios property above. Also, pinctrl-0 defines the pin configuration for the whole SPI module. pinctrl-0 links to pinctrl_ecspi1_1, which can be found in imx6qdl.dtsi and should look like this:

		pinctrl_ecspi1_1: ecspi1grp-1 {
			fsl,pins = <
				MX6QDL_PAD_EIM_D17__ECSPI1_MISO 0x100b1
				MX6QDL_PAD_EIM_D18__ECSPI1_MOSI 0x100b1
				MX6QDL_PAD_EIM_D16__ECSPI1_SCLK 0x100b1
				MX6QDL_PAD_EIM_EB2__GPIO2_IO30  0x000f0b0
				MX6QDL_PAD_KEY_COL2__GPIO4_IO10 0x000f0b0
			>;
		};

This includes the MISO, MOSI and SCLK lines, plus two chip select lines configured as GPIOs.

Pin naming on the i.MX6 can be a bit confusing sometimes. You can find a good explanation of i.MX6 gpios and pin naming at http://www.kosagi.com/w/index.php?title=Definitive_GPIO_guide. Anyway, I’ll try to briefly explain a way to deal with GPIOs in the device tree, for those who are aware of how important it is to understand Wandboard GPIOs for their daily survival. If, on the other hand, you are a normal mentally healthy person, you can safely skip the next section.

GPIO naming on the device tree
In order to name gpios in the device tree, the first thing you need to do is find out which pad you are using. The pad is the actual physical pin on the i.MX6. The chip select pins we are using here are the ones reserved on the expansion connector: CSPI1_CS0 and CSPI1_CS1, as shown in the Wandboard schematic.
spi_pinout

But beware! These names don’t really mean anything. They are just net names chosen for the carrier board of the Wandboard, and are not the pad names at the i.MX6 chip. I don’t know what’s the proper way of finding out the pad names, surely it is documented somewhere, but what I normally do is have a look at the schematics, and “follow” the net. Diving in the schematics a bit we see that the chip select lines are wired straight to the pins 228 and 230 of the EDM connector (the big connector putting the carrier and the module boards together):

spi_edm_carrier

…which doesn’t tell us much. But looking now at the same connector in the system on module schematics (the one with the actual i.MX6 chip on it), we find this:

spi_edm_module

So EIM_EB2 and KEY_COL2 are our new candidates for pad names. Happily, there is a file in the kernel tree with all the pad names and their possible functions: arch/arm/boot/dts/imx6q-pinfunc.h (versions for dual and solo available too). And it looks like EIM_EB2 and KEY_COL2 are right there:

...
#define MX6QDL_PAD_EIM_EB2__EIM_EB2_B               0x08c 0x3a0 0x000 0x0 0x0
#define MX6QDL_PAD_EIM_EB2__ECSPI1_SS0              0x08c 0x3a0 0x800 0x1 0x0
#define MX6QDL_PAD_EIM_EB2__IPU2_CSI1_DATA19        0x08c 0x3a0 0x8d4 0x3 0x0
#define MX6QDL_PAD_EIM_EB2__HDMI_TX_DDC_SCL         0x08c 0x3a0 0x890 0x4 0x0
#define MX6QDL_PAD_EIM_EB2__GPIO2_IO30              0x08c 0x3a0 0x000 0x5 0x0
#define MX6QDL_PAD_EIM_EB2__I2C2_SCL                0x08c 0x3a0 0x8a0 0x6 0x0
#define MX6QDL_PAD_EIM_EB2__SRC_BOOT_CFG30          0x08c 0x3a0 0x000 0x7 0x0
...
#define MX6QDL_PAD_KEY_COL2__ECSPI1_SS1             0x208 0x5d8 0x804 0x0 0x2
#define MX6QDL_PAD_KEY_COL2__ENET_RX_DATA2          0x208 0x5d8 0x850 0x1 0x1
#define MX6QDL_PAD_KEY_COL2__FLEXCAN1_TX            0x208 0x5d8 0x000 0x2 0x0
#define MX6QDL_PAD_KEY_COL2__KEY_COL2               0x208 0x5d8 0x000 0x3 0x0
#define MX6QDL_PAD_KEY_COL2__ENET_MDC               0x208 0x5d8 0x000 0x4 0x0
#define MX6QDL_PAD_KEY_COL2__GPIO4_IO10             0x208 0x5d8 0x000 0x5 0x0
#define MX6QDL_PAD_KEY_COL2__USB_H1_PWR_CTL_WAKE    0x208 0x5d8 0x000 0x6 0x0
...

The syntax here is MX6QDL_<PAD_NAME>__<FUNCTION>, so there are 7 different functions for every pad. Of these, the relevant functions for us are chip select and GPIO:

MX6QDL_PAD_EIM_EB2__ECSPI1_SS0
MX6QDL_PAD_EIM_EB2__GPIO2_IO30

MX6QDL_PAD_KEY_COL2__ECSPI1_SS1
MX6QDL_PAD_KEY_COL2__GPIO4_IO10

ECSPI1_SSx is the name of the native chip select function.

Chip select: native or GPIO
So we have two ways of configuring our chip select pins: native or GPIO.

Here is a summary of how you configure your device tree files for native or GPIO chip select behaviour:

File Native version GPIO version
imx6qdl-wandboard.dtsi cs-gpios = <0>, <0>; cs-gpios =
<&gpio2 30 0>, <&gpio4 10 0>;
imx6qdl.dtsi MX6QDL_PAD_EIM_EB2__ECSPI1_SS0
MX6QDL_PAD_KEY_COL2__ECSPI1_SS1
MX6QDL_PAD_EIM_EB2__GPIO2_IO30
MX6QDL_PAD_KEY_COL2__GPIO4_IO10

I would recommend using the GPIO configuration. I had trouble with the native chip select, where the chip select line would toggle between bytes in a transfer, where you would expect it to remain low.

Native chip select: Chip select (yellow) toggles between bytes
Native chip select: Chip select (yellow) toggles between bytes
GPIO chip select: Chip select remains low for the whole transfer
GPIO chip select: Chip select remains low for the whole transfer

These two results happen with the same code (using a SPI_IOC_MESSAGE ioctl on the spidev node).

Almost done!

Anyway, time to round up! Compile the device tree with:

$ make imx6dl-wandboard.dtb

or

$ make imx6q-wandboard.dtb

and copy it to your boot folder. (Again, the Wandboard solo should follow a similar approach). Reboot, modprobe spidev, and check you have your /dev/spidev1.0 and /dev/spidev1.1 files.

Don’t worry if they have a different bus number (in my case they were called /dev/spidev32766.x), the important thing is the pin configuration in the device tree is good.

Hope this helps. I wrote this to the best of my knowledge but I am not a guru of SPI at all, so please let me know of any correction!

5 Comments