September 25, 2022
tensorflow keras c plus plus

TensorFlow-Keras Model, Train in Python, Use in C++

Despite many advantages of Python such as simple syntax, easy-to-read code and large number of available 3rd-party packages, when it comes to embedded systems and real-time applications, C++ is undoubtedly one of the most efficient programming languages that dominates the realm of these systems.

Embedding System
Top 10 languages that used in embedded projects (Reference)

However, complex syntax of C++ makes it difficult to implement high level algorithm such as Machine Learning algorithms. For example building a small CNN network and it’s training rail would be so complex that no developer thinks about doing it in C++ from the scratch.

In this post, we will learn how to transfer and use CNN models that is trained in Python with Keras to C++ using a light weight header-only library named frugally-deep. I found no tutorial on the internet that explain how to use this library in details. Here I am going to explain it step by step.

First we train a simple CNN network in Python on a 2000 Cat & Dog image dataset. No matter how much accuracy we achieve, we just save the model :

import cv2
import glob
from sklearn.neighbors import KNeighborsClassifier
import numpy as np
from sklearn.preprocessing import LabelEncoder
from tensorflow.keras.utils import to_categorical
from joblib import dump, load
from sklearn.model_selection import train_test_split
import tensorflow as tf
from tensorflow.keras import models
from tensorflow.keras import layers
import os
for item in glob.glob("Dataset/*/*"):
img = cv2.imread(item,cv2.IMREAD_GRAYSCALE) # read images with one channel grayscale
r_img= cv2.resize(img,(128,128)) # resize to 128x128
r_img = np.expand_dims(r_img, axis = -1)
data.append(r_img) # add resized image to dataset list
label = item.split("\\")[1]
labels.append(label) #add image label to dataset list
le = LabelEncoder()
labels = le.fit_transform(labels)
labels = to_categorical(labels)
data = np.array(data)/255 # Normalize channel between 0 to 1
#split test and train randomly
x_train, x_test, y_train, y_test = train_test_split(data,labels,test_size=0.2)
net= models.Sequential(
net.compile(optimizer="SGD", loss="binary_crossentropy",metrics=["accuracy"])
H =,y_train,batch_size=32, epochs=24, validation_data=(x_test,y_test))"CatDogNew.h5") # Save the model
view raw hosted with ❤ by GitHub

After running the above code , we will have our model “CatDogNew.h5” file saved in project folder.

Before we starting the next step we need to download the library from its repository on gihub. After download , find “keras_export” directory in the main folder of the library, and open command prompt from this directory :

keras model export to json
open command prompt in keras_export folder

Now, we convert the model to json file using . Run the following command in the command line to build model in json format :

python path/to/your/model/CatDogNew.h5 CatDogNew.json

Running the command will yield “CatDogNew.json” in the keras_export folder.

In this step which is so important, we prepare our Visual Studio project to use the frugally-deep library. I use Visual Studio 2019, however there is no big difference between this version and its old ones. First, Create new Console App and choose a name for it.

keras model to cpp json
Create new Console App project

After creating the Project, you have one cpp file opened with a main function containing “hello world” output. Now, It’s time to include library header file in project. Frugally-deep depends on 3 other libraries which they are also header-only and can be included without any pain. The libraries are :

  • Eigen
  • fplus
  • nlohmann

I gathered all of them in a zip file that you can download it with just a click here. I added the frugally-deep header files to the zip file so you don’t need to get it from source code downloaded from github.

Extract zip file and move the include folder to a place that is easy to address later. The include folder must contains all libraries like below :

the extracted folder should contain 4 libraries

Open project properties from Project menu, in left side menu click on VC++ Directories, then in right edit the Include Directories row and insert the path to the extracted include folder containing 4 libraries in the field :

keras model to cpp
Add extracted include folder address to project

As the the frugally-deep is a header-only library, it need no more configuration in project properties. So, we can go to the project’s main file and past following codes in it :

#include <iostream>
#include <fdeep/fdeep.hpp>
#include <opencv2/core/core.hpp>
#include <opencv2/highgui/highgui.hpp>
#include <opencv2/imgproc.hpp>
#include <filesystem>
#include <exception>
using namespace stdext;
namespace fs = std::filesystem;
int main()
std::string test_image_folder_path = "d:/catdog/"; // path to folder containing test images
const auto mymodel = fdeep::load_model("D:/CatDogNew.json"); // load the converted model
for (const auto& entry : fs::directory_iterator(test_image_folder_path)) {
std::string image_path = entry.path().string(); // get images path one by one
const cv::Mat image = cv::imread(image_path, cv::IMREAD_GRAYSCALE); // read the image in single channel grayscale mode
cv::Mat resizedImg;
cv::resize(image, resizedImg, cv::Size(128, 128)); // resize image to the 128x128 (Model input dimension)
imshow("Display Window", resizedImg); // just show the image
// convert cv::MAT to fdeep::tensor
const auto input = fdeep::tensor_from_bytes(resizedImg.ptr(),
0.0f, 1.0f);
auto result = mymodel.predict({ input }); // predict the image's label and ouput a 1x2 tensor containing each class probability
std::cout << fdeep::show_tensors(result) << std::endl; // print the tensor
catch (const std::exception& e)
std::cout << e.what() << '\n';

lets see what happens in important parts of the code, line by line:

Line 1 to 8 is doing include jobs. We need to have opencv installed, to install and use opencv in c++ please see this post. Then we load the json model in line 16. We did 50% of the work, but the important part of the code is remained, reading image and applying preprocess. We must feed the model with (128,128) single channel image that is normalized to [0-1] for each pixel. In line 21 we read the image in grayscale mode, then resize it in line 24. The normalization part still remained. We will do it when we are converting the image matrix to tensor. The tensor_from_bytes function in frugally-deep library does it for us in line 28. It takes 6 input parameters, first one is the image matrix pointer, the second and third parameters are the image height and image width. Fourth parameter is the number of channels. And very important last parameters are intended to rescale the channel values from range [0, 255] to [low, high] in which the low and high must be specified with fifth and sixth parameters.

Finally we give the input tensor to predict function, maybe you ask why we give it within {}, answer : frugally-deep support multi-input models, so we give inputs in {} and separated by comma. The predict function output a tensor array that each element shows the corresponding class probability.

Final Output that predicted the Cat with 58% of confidence

Congratulations! You have made your way to the end of this short journey. Our CNN model was trained in python, converted, and then used in C++.

Thank you so much for reading this post!

8 thoughts on “TensorFlow-Keras Model, Train in Python, Use in C++

  1. That’s wonderful for me learning about relationship between Python and C++.
    Thank you very much.
    If you can display code work with RGB image, that will more helpful for me and other readers.

  2. First of all, thank you for posting a good article.
    I want to ask you any problem.
    What is causing the error when model load ?

    model full_model() <– model.hpp

  3. Hi,
    Is it faster than running the model with Python ?
    If so, do you have any idea on how much time do you gain from doing that ?

    Jean-Luc moosh

Leave a Reply

Your email address will not be published. Required fields are marked *