Computer Vision Machine Learning for Orange PI and Rock 5 SBC

Computer Vision based on Machine Learning of Artificial Intelligence is a very promissing application area of Orange Pi Single Board Computers (SBC). Intelligent surveilance and monitoring functions can be embedded into Orange Pi applications, achieving edge computing. To see what applications you can develop with computer vision, you need to understand what computer vision models can do. It is important to understand that current computer vision technology has limitations. So limited applications are possible. Common computer vision modeling types include;

• Image classification: Given an image, this provides classification information. This gives probability of each object class. For example, bird/35%, car/15%, horse/5%, etc. Of course, object with the highest probability is the object class/type of the image.
• Image regression: Given an image, this gives single or multiple numerical output values. For example, probability of being cancerous, temperature, left/right moves, etc.
• Object detection: Given an image, this gives bounding box information of detected objects, such as probability, X/Y coordinates, width, height. It can also provide detected object class/type.
• Similarity regression of two images: Given two images, this gives (whatever) probability of two images. One such application can be face recognition whether two images are of the same person.
• Stereoscopic regression such as distance measurement.

Computer Vision Modeling

Convolutional Neural Networks (CNN) are used to model computer vision tasks. To do computer vision modeling, fairy good understanding of CNN is essential. Computer vision model development involves the following stages;

• Image data collection and preparation.
• CNN model network configuration.
• Model training.
• Model testing.
• Deploy into edge computing applications such as Orange Pi.

Computer vision machine learning is a very complex process. You need powerful but still easy to use and learn Machine Learning Software. CMSR Machine Learning Studio is a fully GUI based machine learning model development platform. You don't code anything until you embed CMSR generated model codes into your application projects. You just need to call a model function from your main program. Users can train machine learning models without coding. It's very easy to use and comes with powerful features. It provides the following types of computer vision modeling tools;

• CNN: Convolutional neural network for image classification and class probability.
• FCN: Fully convolutional network for image classification and class probability. Same as CNN.
• M-CNN: Multi-value output CNN. It's a regression modeling algorithm.
• OD-CNN: Object detection CNN. It detects objects and provides bounding box information. This is very similar to YOLO. You can develop your own YOLO models with this.
• T-CNN: Twin CNN for similarity prediction for such as face recognition.
• S-CNN: (Experimental) Measures distance from stereoscopic images.

Powerful GPU and Large RAM Memory Computer is Essential!

Computer vision is extremely compute intensive. Especially training will take huge computing time on powerful computers. The following shows times taken to train 104 layer computer vision deep neural network with 77 convolution layers on various GPUs. This is one epoch training time with 3,400 68x68 images.

• OPi5 Rk3588 4 eu 64 cores 1000MHz : 1h 16m 11s (76m)
• Toshiba Intel i5 internal 24 cu 900MHz : 2h 23m 40s (143m)
• MacMini Intel i5 internal 40 cu 1100MHz : 1h 17m 32s (77m)
• Acer Intel i7 internal 24 cu 1150MHz : 1h 1m 12s (61m)
• Acer Nvidia Quardro T1000 896 cores 1725MHz : 8m 40s (8m)

You can see even high-end Intel internal GPUs can barely beat Orange Pi 5 GPU! Nvidia external GPU is outstanding. That's why everyone is rushing to buy Nvidia GPUs.

More shading GPU cores are better as CMSR Studio can take advantage of bigger shading cores such as Nvidia Cuda cores. CMSR employes fine grained data parallelism. On 896 Nvidia Cuda cores, we observed 165 times fast than single CPU core model training. With more Cuda cores, it can get over 1,000 times fast. If Orange Pi 5 GPU is too slow for your projects, high performance gaming Windows PC with powerful external GPU from Nvidia or AMD is what you need. Note that CMSR Data Miner/Machine Learning Studio runs on Orange Pi 5 Linux distros with GPU acceleration. You can do both data analysis and computer vision model development on Orange Pi 5 as well as on Windows and MacOS.

Model training is done with randomized order images. Otherwise, models will develop skews towards later images. To read images randomly, all training data images must be brought into main memory. Otherwise training will be extremely slow. You can estimate needed total RAM memory size in bytes with the following formula;

   total image dataset size = ((image width) * (image height) * 3 + 3) * (number of images)

This should be your maximum image dataset size. For your computer RAM, it should be about twice of this size as OS will also use RAM. If you don't have large memory, you will have to content with small image training datasets. Note that this is CPU RAM size. GPU VRAM size is different. It can be much smaller as GPU VRAM stores only model parameters and some extras.

Model Code Generation for Embedded Applications

Forget about ChatGPT thing! CMSR ML Studio can generate highly efficient AI ML codes.

CMSR ML Studio provides easy embedding into applications. Just generate program codes and compile with your project codes. Just call a function from your main code. CMSR can generate the following type codes;

• Single CPU thread: Java, C, Swift.
• Multicore CPU: C++, Java.
• GPU: OpenCL (Java, C++), Cuda (Java, C++), OpenGL ES3 (C++), Metal (Swift, Objective-C).

generated programs incorporate pre- and post processing as follows;

• Color inversion.
• Color transformation and value encoding.
• Histogram equalization.
• Object filtering and duplicate removal.
• Sorting.

For a generated program example, please see CMSR Generated Program Example: C++ OpenCL.

Python developers will need to use Python-C++ bindings. Pure Python codes will be 80 times slower than C++ codes. So it's pointless to develop pure Python code generation. If OpenCL is supported, C++ OpenCL is recomended. Otherwise multicore version should be used.

Generated Code Performance on Orange Pi 5

Orane Pi is very impressive. It's very suitable for edge computing and computer vision. We tested 25 million parameter object detection computer vision model on Raspberry Pi 4B and Orange Pi 5 SBC computers. Results are as follows;

• Raspberry Pi 4B: 27 seconds (using 4 CPU threads).
• Orange Pi 5: 14.5 seconds (using 4 CPU threads).
• Orange Pi 5: 13 seconds (using 8 CPU threads).
• Orange Pi 5: 0.68 seconds (using OpenCL GPU).

Orange Pi 5 OpenCL GPU performance is very impressive. It's suitable for large computer vision models. When 4 or 8 CPU threads are used, average power consumption is 11 watts during 13 or 14.5 seconds. When OpenCL GPU is used, average power consumption is only 8 watt during 0.68 second. GPU is more than 26 times more power efficient than CPUs. So the winner is Oprange Pi 5 OpenCL GPU. Note that you can enable OpenCL on Debian and Ubuntu for Orange Pi 5. Installation details can be found from the following link;

• Install OpenCL on Debian and Ubuntu and Armbian for Orange Pi 5

You may also need OpenCV to capture camera images in RGB/BGR format. Installing OpenCV on OPi5 on Ubuntu and Debian, you may face two problems. One problem is that it requires 8GB main memory space to compile some modules. If your Opi5 has 4GB RAM, it will have 1.8GB swap space. Expand swap space to have additional 4GB as described here. Note that use "sudo fallocate -l 4G /swapfile" instead of "sudo fallocate -l 1G /swapfile" in the link. The other problem is that PNG library in your system may not be compatible with OpenCV. To avoid this problem, install OpenCV as described here. But use "cmake -DBUILD_PNG=ON ../opencv" command instead of "cmake ../opencv" described in the link.

Embedding Models into Applications

CMSR generated codes are of very high efficiency. Especially in edge computing, efficiency and speed is one of the most important factors. All you need to code is what you actually use them. The following code shows usage of CNN classification model in C++. You will create a couple of arrays to receive results. Then call the main evaluate function with parameters. You can repeat "evaluate" function as much as you need in your applications. Of course, your applications should get image data from onboard camera!

#include <iostream>
#include "CMSRModel.hpp"
using namespace std;

int main(void) {
	char filename[] = "data/modelfile.cnn";
	char imagefile[] = "data/cnnimages.rgb";

	int IMAGEARRAY[64*64*3];
	int outLabelCount = 4;
	int outLabelIndices[5];
	float outLabelProbabilities[5];
	int blackandwhite = 0;
	int r0g1b2 = 1;

	CMSRModel *model = new CMSRModel();
	model->verbose = true;

	// initialize model;
	model->initializeModel(4, filename);

	// The following steps can be repeated many times;
	model->populateImageArray((int*)IMAGEARRAY, imagefile, 64*64*3); // you can get data from camera!
	model->evaluate (
		outLabelCount, /* result label count */
		outLabelIndices, /* ordered result output label indices */
		outLabelProbabilities, /* ordered result output label relative probabilities */
		blackandwhite, /* 1 if black and white, otherwise 0 */
		r0g1b2,        /* 1 if IMAGEARRAY[][][0] is red, otherwise 0 */
		IMAGEARRAY   /* [row/height][column/width][colors] */
	);
	cout << "Results;\n";
	for (int i=0; i < outLabelCount; i++) {
		cout << i << ": " << outLabelIndices[i] << " / " << outLabelProbabilities[i] << "\n";
	}

	// release memory resources;
	model->releaseMemoryResources();
	delete model;

	cout << "End.\n";

	return 0;
}