In this video, we will start with the image
processing series which uses OpenCV along
with a raspberry pi. I find image processing
very fascinating and it is not as complicated
as it first appears to be. By the end of this
video, you will get a better understanding
of the pixels that make up an image, you will
learn how to install OpenCV on a raspberry
pi and we will also write some test scripts
to capture an image and then display the individual
red, green and blue color components. So let’s
dive in.
First, let’s talk about OpenCV. OpenCV is
an open source computer vision and machine
learning software library. It is perhaps the
most popular tool used for image and video
processing due to its vast number of optimized
algorithms.
Before we actually start setting it up and
testing it, let’s talk about images, in
particular the pixels that make up images.
Every digital image is made up of individual
pixels, the number of which being dependent
on the image resolution. A 1920x1080 image,
will have 1920 pixels along its width and
1080 pixels along its height. Each of these
pixels represents a colour that is made up
of Red, Green and Blue components. These days,
most of the displays use colored LEDs to create
these color components.
Here I have a small display connected to the
raspberry pi. I’m going to use this USB
microscope to focus on the top left corner
here and I’m going to zoom in to show you
the individual LEDs. If I move the mouse around
the area, you will see the LEDs switch ON/OFF
depending on the final color that needs to
be produced. Packing these individual LEDs
or pixels closer together results in much
higher image quality as it’s a bit more
difficult to distinguish the individual LEDs
In order to represent the image digitally,
we need to know the position of the pixel
and then the values of the individual RGB
components. All this information is represented
by numbers and once we have this information,
we can carry out all sorts of mathematical
operations on it which is what image processing
is all about. A video is simply a series of
images taken in sequence and this means that
it applies to videos as well. We will learn
more about this in future videos, so let’s
get started.
The first thing we need to do is download
an operating system for the raspberry pi.
For this project, we will be using the Desktop
version as we need the GUI components in order
to view the images. We then need to flash
this image to the microSD card using Etcher.
The board needs to connect to the internet
in order to download and install the libraries
and you can either plug in an ethernet cable
or use the onboard WIFI.
You can connect a display along with a keyboard
and mouse to do this, but I prefer using a
single computer for everything and that is
why I will be using a remote desktop connection
for this video. There’s a video on this
topic which goes into further details and
I’d recommend watching that if this is all
new to you.
I then copy the wpa_config file to give the
board network access and I also copy the ssh
file so that we can remotely control it. Once
done, I simply insert the microSD card, connect
the camera and power ON the board. In order
to SSH into the board, we would need it’s
IP address and that can be obtained by using
AngryIP scanner. Once that’s done, you can
simply open up a command prompt window and
ssh into the board. I will first enable remote
access by installing tightvncserver and then
xrdp. Once completed, I can simply use a remote
desktop connection to log in to the board.
Here’s what we need to do in order to install
OpenCV from scratch. We first need to enable
and test the camera, update the operating
system, check and install pip if needed, install
the additional libraries required for OpenCV
along with opoenCV itself, and finally we
need to install the picamera module before
we can use it in OpenCV.
You can enable the camera by running the sudo
raspi-config command. You can then navigate
into the interfacing options and then enable
the camera. You will need to press the TAB
key to navigate to the Finish option and then
reboot the board. Give the board a minute
and then log back in. It’s a good idea to
test if the camera is working and that can
be done by running the following command.
This will capture an image and save it to
a file. We can then use the file browser to
open it which indicates that everything is
setup correctly.
The operating system can be updated by running
the following command. This step may take
some time, depending on your network connection.
Just like the last video, we will be making
use of pip or the package installer for python.
It should be installed by default but you
can run the following command to verify and
install it if needed. Pip3 is the package
installer for python version 3 which is currently
the recommended version to use. We then need
to install OpenCV itself and there are a few
commands that need to be run before this.
I’d recommend opening up the written post
from our website using the raspberry pi so
that you can simply copy and paste the commands
like so. These commands install all the dependencies
along with OpenCV itself. We are almost ready
to write scripts now but before we do that,
we need to install the picamera module which
can be done by running this command. We need
to install the array version of this module
as that makes processing the captured images
much simpler. Remember that the image is represented
by matrices and we will be carrying out mathematical
operations on them.
I use the term matrix here which is a type
of an array representation where numbers are
arranged in rows and columns. If this sounds
complicated, then simply think of them as
arrays where the numbers collectively represent
the final image. You can read more about it
in the reference documentation.
Once all of this is done, we can create a
new script file called test-opencv.py to test
if everything is working as it should. Once
again, I would recommend copying the script
contents directly from the website. The script
starts by importing the relevant modules.
We have the picamera module as usual along
with the piRGBarray module which allows us
to obtain the 3-dimensional array of the image.
We import the time module as we will be adding
a delay. And finally, we import cv2 which
is the OpenCV module.
The actual script starts by referencing the
picamera module and setting up the imageCapture
variable to store the RGB array version of
the image. We then call a 100ms delay to let
the camera white balance settle but this is
optional. We then capture an image form the
camera and the array format is BGR or blue,
then green then red. This is specific to OpenCV
and it is recommended that you stick to it
to avoid bugs and issues later on.
We then reference the image again and display
it using the imshow function. This function
allows you to give a title to the window and
this can be updated as needed. The final line
waits for the user to press a key before exiting
the script. If this line didn’t exist, then
the script would terminate immediately, even
before we get the chance to analyse the image.
This script can be run by typing in python3
test-opencv.py. Remember that we are using
python3 here and that’s why we need to type
the number 3. The script captures an image
and displays it in the window with the title
specified. You also get the XY pixel position
and it’s RGB values. These get updated as
I move the cursor around. Remember that we
need to press a key on the keyboard in order
to exit the script. Closing the window will
not exit the script, so keep that in mind.
Before we conclude this video, let’s try
to manipulate the image a bit, for that, we
need to first get a better understanding of
how images are stored. In simple terms, here’s
how the image is represented. We have a 3
dimensional array that can be represented
as x,y, c and v. The x & y values, give us
the pixel location as we saw in the previous
example. We then have three different values
for each of the components or channels, this
is given by v. The value of C helps us determine
if we are talking about the RED, GREEN or
BLUE components. OpenCV stores images in the
bgr format, which means that a C value of
0 indicates the blue component, 1 represents
the green component and 2 represents the red
component.
Now that we have this information, it is easy
for us to obtain the individual color components
of the image and that is what we will do next.
Simply create a new script called image-components.py
and then copy the script from the website
post. All we’ve done here is created a copy
of the image called b which represents the
blue component. We then set the green and
red channels to 0. Remember that the C value
of 1 and 2 represents the green and red components
respectively. We then repeat this for the
other two channels. This is not the most efficient
way to do it but it helps you understand what
is going on at a basic level. When manipulating
images, it is very common to create copies
of the original image so that you still have
the source which can be used for future computation.
We then simply display the individual components
by creating 3 different windows with the correct
name for reference. If you run the script
then you will be able to view an output like
this. Moving the mouse around will show you
that the individual images contain a single
color component – starting with the red,
green and then blue. In other words, the original,
captured image is made up of these color components
and because the pixels are so close together,
we do not notice the individual colors but
only notice the final result.
So that’s a very basic introduction to installing
and using OpenCV on the raspberry Pi. We will
get into more details in future videos along
with some level of machine learning. The image
processing videos will be released on Sunday
so please do subscribe to get notified. Thank
you for watching and I will see you in the
next one!
