Unreal Engine: Including a Third-Party Library (on the example of the Point Cloud Library and Boost) [Tutorial] [Download]

I recently tried to include the Point Cloud Library into Unreal. I ran into so many problems and had such a hard time finding proper answers/solutions (since Unreal’s documentation is kinda crappy regarding third party plugin creation) that I was close to burn Epic’s headquarters and once I finally figured things out, decided to make a small tutorial about including an external C++ library (on the example of the Point Cloud Library (PCL) and boost) into the Unreal Engine. I hope this’ll help some folk and will help prevent Epic’s headquarters to be burned in the future. 😀

General information:

I used the Unreal Engine 4.16, Visual Studio 2017, PCL 1.8.1 and boost 1.64; all 64-bit.

Download:

If you’re too lazy to read and just want a proper starting point for your third party plugin or just want to use boost/PCL functions in your project, you can download the template here:
https://github.com/ValentinKraft/Boost_PCL_UnrealThirdPartyPlugin


Including PCL & Boost into Unreal:

Before including PCL into Unreal, let’s start with doing the basic infrastructure, by creating an Unreal plugin. This plugin will consist out of two modules: The „PointCloudRenderer“ and the „PCL“ module. The external „PCL“ module will handle the import of the Third-Party libraries (= boost/PCL headers and libraries), while the „PointCloudRenderer“ will contain our core implementation. (Later, we will also create an interface module to call functions from the editor, see „Create a Connection to your Plugin in the Editor“). For making a plugin, I’d recommend you to use the built-in wizard (under Plugins –> New). Additionally, the documentation might help you with this (I decided for a „Non-Game Module“):
https://wiki.unrealengine.com/An_Introduction_to_UE4_Plugins
and/or here:
https://puppet-master.net/tutorials/unreal-engine-4/create-an-engine-plugin-using-blankplugin/

Inside of your just created Plugin’s Source folder you now have to create a „ThirdParty“ folder. Here you have to insert all the headers and libraries you want to include. For a minimal PCL project, you probably need the „Boost“ library and headers, the „Eigen“ library and the PCL headers and following libraries (if you do not know how to create the header/lib-files, have a look at the „Building PCL / Where to get the right .lib/.dll files from?“ section):

  • pcl_common_release.lib
  • pcl_io_release.lib
  • pcl_io_ply_release.lib
  • pcl_octree_release.lib
  • pcl_ml_release.lib

I chose the following folder structure for the third party assets:

  • [PluginDir]/Source/ThirdParty/[ThirdPartyLibName]/lib for the libraries and
  • [PluginDir]/Source/ThirdParty/[ThirdPartyLibName]/include for the headers.

Now you have to include the libraries into the Unreal/Visual Studio project. In Unreal, the UnrealBuildTool handles the import of all dependencies, so we have to tell the UnrealBuildTool to include the new libraries and headers via the corresponding .Build.cs-file. In your project, there should be already a „[PluginName].Build.cs“ (in my case, a PointCloudRenderer.Build.cs). What’s missing is the .Build.cs-file for our PCL module (we could also use the PointCloudRenderer.Build.cs, but its cleaner to do it in two separate Build-files), so you have to create a PCL.Build.cs with the following content:

// Copyright 1998-2017 Epic Games, Inc. All Rights Reserved.

using System.IO;
using System;

namespace UnrealBuildTool.Rules
{
    public class PCL : ModuleRules
    {
        private string ModulePath
        {
            get { return ModuleDirectory; }
        }

        private string BinariesPath
        {
            get { return Path.GetFullPath(Path.Combine(ModulePath, "../Binaries/")); }
        }

        public PCL(ReadOnlyTargetRules Target) : base(Target)
        {
            // Tell Unreal that this Module only imports Third-Party-Assets
            Type = ModuleType.External;

            LoadPCL(Target);
        }

        public bool LoadPCL(ReadOnlyTargetRules Target)
        {
            bool isLibrarySupported = false;
            bool bDebug = (Target.Configuration == UnrealTargetConfiguration.Debug && BuildConfiguration.bDebugBuildsActuallyUseDebugCRT);

            if (Target.Platform == UnrealTargetPlatform.Win64)
            {
                isLibrarySupported = true;

                //string PlatformString = (Target.Platform == UnrealTargetPlatform.Win64) ? "x64" : "x86";

                // Explicitly name the used libraries
                PublicAdditionalLibraries.Add(Path.Combine(ModulePath, "Boost/lib/libboost_chrono-vc141-mt-1_64.lib"));
                PublicAdditionalLibraries.Add(Path.Combine(ModulePath, "Boost/lib/libboost_date_time-vc141-mt-1_64.lib"));
                PublicAdditionalLibraries.Add(Path.Combine(ModulePath, "Boost/lib/libboost_filesystem-vc141-mt-1_64.lib"));
                PublicAdditionalLibraries.Add(Path.Combine(ModulePath, "Boost/lib/libboost_iostreams-vc141-mt-1_64.lib"));
                PublicAdditionalLibraries.Add(Path.Combine(ModulePath, "Boost/lib/libboost_system-vc141-mt-1_64.lib"));
                PublicAdditionalLibraries.Add(Path.Combine(ModulePath, "Boost/lib/libboost_thread-vc141-mt-1_64.lib"));

                PublicAdditionalLibraries.Add(Path.Combine(ModulePath, "PCL_181/lib/pcl_common_release.lib"));
                PublicAdditionalLibraries.Add(Path.Combine(ModulePath, "PCL_181/lib/pcl_io_release.lib"));
                PublicAdditionalLibraries.Add(Path.Combine(ModulePath, "PCL_181/lib/pcl_io_ply_release.lib"));
                PublicAdditionalLibraries.Add(Path.Combine(ModulePath, "PCL_181/lib/pcl_octree_release.lib"));
                PublicAdditionalLibraries.Add(Path.Combine(ModulePath, "PCL_181/lib/pcl_ml_release.lib"));
                //PublicAdditionalLibraries.Add(Path.Combine(ModulePath, "PCL_181/lib/pcl_visualization_release.lib"));
            }

            if (isLibrarySupported)
            {
                PublicIncludePaths.Add(Path.Combine(ModulePath, "Eigen/eigen3"));
                PublicIncludePaths.Add(Path.Combine(ModulePath, "Boost/include/boost-1_64"));
                PublicIncludePaths.Add(Path.Combine(ModulePath, "PCL_181/include/pcl-1.8"));
                //PublicIncludePaths.Add(Path.Combine(ModulePath, "FLANN/include"));
                //PublicIncludePaths.Add(Path.Combine(ModulePath, "VTK/include/vtk-8.0"));

                // Not sure if needed
                Definitions.Add("_CRT_SECURE_NO_WARNINGS=1");
                Definitions.Add("BOOST_DISABLE_ABI_HEADERS=1");

                // Needed configurations in order to run Boost
                bUseRTTI = true;
                bEnableExceptions = true;
                //bEnableUndefinedIdentifierWarnings = false;
            }

            Definitions.Add(string.Format("WITH_PCL_BINDING={0}", isLibrarySupported ? 1 : 0));
            Definitions.Add(string.Format("WITH_BOOST_BINDING={0}", isLibrarySupported ? 1 : 0));

            return isLibrarySupported;
        }
    }
}

And in the Plugin’s Build.cs file you just have to add your newly created „PCL“ module as a „PublicDependencyModule“:

// Copyright 1998-2017 Epic Games, Inc. All Rights Reserved.

using System.IO;
using System;

namespace UnrealBuildTool.Rules
{
    public class PointCloudRenderer : ModuleRules
    {
        private string ModulePath
        {
            get { return ModuleDirectory; }
        }

        private string ThirdPartyPath
        {
            get { return Path.GetFullPath(Path.Combine(ModulePath, "../ThirdParty/")); }
        }

        private string BinariesPath
        {
            get { return Path.GetFullPath(Path.Combine(ModulePath, "../../Binaries/")); }
        }

        public PointCloudRenderer(ReadOnlyTargetRules Target) : base(Target)
        {
            PrivateIncludePaths.Add("Source/PointCloudRenderer/Private");
            PublicIncludePaths.Add("Source/PointCloudRenderer/Public");

            PublicDependencyModuleNames.AddRange(new string[] { "Core", "CoreUObject", "Engine", "InputCore", "PCL" });
            PrivateDependencyModuleNames.AddRange(new string[] { "Core" });

            // Since the PCL module needs this, we also have to use these flags here
            bUseRTTI = true;
            bEnableExceptions = true;
            //bEnableUndefinedIdentifierWarnings = false;
        }
    }
}

And don’t forget to update the module section in your .uplugin-File:

{
	"FileVersion" : 3,
	"Version" : 1,
	"VersionName" : "1.0",
	"FriendlyName" : "Point Cloud Renderer",
	"Description" : "SCIENCE, BITCH.",
	"Category" : "Rendering",
	"CreatedBy" : "Valentin Kraft",
	"CreatedByURL" : "http://www.valentinkraft.de",
	"DocsURL" : "",
	"MarketplaceURL" : "",
	"SupportURL" : "",
	"EnabledByDefault" : false,
	"CanContainContent" : false,
	"IsBetaVersion" : true,
	"Installed" : false,
  "Modules": [
    {
      "Name": "PointCloudRenderer",
      "Type": "Runtime",
      "LoadingPhase": "PreDefault"
    }
  ]
}

Now you need to rebuild the Visual Studio project files using the Unreal build tool (for that, right-click on the uproject-File and choose „Generate Visual Studio project files“ or, when using the Unreal source version from Github, using the GenerateProjectFiles.bat). The Unreal build tool will include your PCL dependencies now into your plugin. And there you go! PCL is included in your plugin. Now we can include PCL/boost headers and call their functions from within Unreal (see line 46):

// (c) 2017 by Valentin Kraft, http://www.valentinkraft.de

#include "CoreMinimal.h"
#include "Modules/ModuleManager.h"
#include "IPointCloudRenderer.h"
#include "GenericPlatform.h"
#include "GenericPlatformProcess.h"

THIRD_PARTY_INCLUDES_START
#include <pcl/io/pcd_io.h>
#include <pcl/point_types.h>
#include <boost/shared_ptr.hpp>
THIRD_PARTY_INCLUDES_END

DECLARE_LOG_CATEGORY_EXTERN(PointCloudRenderer, Log, All);
DEFINE_LOG_CATEGORY(PointCloudRenderer);

class FPointCloudRenderer : public IPointCloudRenderer
{
public:
	/** IModuleInterface implementation */
	virtual void StartupModule() override;
	virtual void ShutdownModule() override;

	int LoadPCDFile();

private:
	pcl::PointCloud<pcl::PointXYZ> cloud;
};

IMPLEMENT_MODULE(FPointCloudRenderer, PointCloudRenderer)


void FPointCloudRenderer::StartupModule()
{
	UE_LOG(PointCloudRenderer, Log, TEXT("################################### \n"));
	UE_LOG(PointCloudRenderer, Log, TEXT("## Starting Point Cloud Renderer... \n"));
	UE_LOG(PointCloudRenderer, Log, TEXT("################################### \n"));

	FPointCloudRenderer::LoadPCDFile();
}


int FPointCloudRenderer::LoadPCDFile() {

	if (pcl::io::loadPCDFile<pcl::PointXYZ>("C:\\Users\\Valle\\Dropbox\\Uni\\Masterarbeit\\Daten\\bunny.pcd", cloud) == -1)
	{
		UE_LOG(PointCloudRenderer, Warning, TEXT("Couldn't read file test_pcd.pcd \n"));

		return (-1);
	}

	auto w = FString::FromInt(cloud.width * cloud.height);
	UE_LOG(PointCloudRenderer, Log, TEXT("Succesfully loaded %s data points."),*w);

	return (1);
}


void FPointCloudRenderer::ShutdownModule()
{
	// This function may be called during shutdown to clean up your module.  For modules that support dynamic reloading,
	// we call this function before unloading the module.
}

To now call our plugin’s functions from another script/class, we have to add the plugin’s functions to the plugin’s public IModuleInterface (for details see the full repo at Github). After including the plugin’s interface header file (in my case „IPointCloudRenderer.h“) in your script/class, we can now access our plugin’s functions simply via:

if (IPointCloudRenderer::IsAvailable())
	IPointCloudRenderer::Get().LoadPCDFile();

(Further information on linking libraries in Unreal can be found here):
https://wiki.unrealengine.com/Linking_Static_Libraries_Using_The_Build_System
and here:
https://wiki.unrealengine.com/How_to_Link_External_C_Libraries_.dll_.lib_With_Your_Project_%26_Package_With_Game,_Fast_And_Easy


Create a Connection/Component to your Plugin in the Editor:

We can now access our plugin from other scripts, hurray! But how do we include our new functionality into the Unreal editor? For example, it would be nice to have the plugin as a component with some public values we can change from the Editor just like this:

For this, we have to create a custom component. I’d recommend to, again, make a separate module that handles the communication between the Unreal Editor and our (already created) Plugin. For this, follow the instructions above (or, again, use the built-in wizard for Plugins under Plugins –> New). This module we will name „PointCloudRendererEditor“. Your file structure should now look like this:

First, update the .uplugin file with your new module:

{
	"FileVersion" : 3,
	"Version" : 1,
	"VersionName" : "1.0",
	"FriendlyName" : "Point Cloud Renderer",
	"Description" : "SCIENCE, BITCH.",
	"Category" : "Rendering",
	"CreatedBy" : "Valentin Kraft",
	"CreatedByURL" : "http://www.valentinkraft.de",
	"DocsURL" : "",
	"MarketplaceURL" : "",
	"SupportURL" : "",
	"EnabledByDefault" : false,
	"CanContainContent" : false,
	"IsBetaVersion" : true,
	"Installed" : false,
  "Modules": [
    {
      "Name": "PointCloudRenderer",
      "Type": "Runtime",
      "LoadingPhase": "PreDefault"
    },
    {
      "Name": "PointCloudRendererEditor",
      "Type": "Runtime",
      "LoadingPhase": "Default"
    }
  ]
}

The PointCloudRendererEditor’s Build.cs should now include your PointCloudRenderer module (line 23):

using UnrealBuildTool;

public class PointCloudRendererEditor : ModuleRules
{
	public PointCloudRendererEditor(ReadOnlyTargetRules Target) : base(Target)
	{
		PCHUsage = PCHUsageMode.UseExplicitOrSharedPCHs;

        PublicIncludePaths.Add("PointCloudRendererEditor/Public");
        PrivateIncludePaths.Add("PointCloudRendererEditor/Private");

        PublicDependencyModuleNames.AddRange(
                new string[]
                {
                    "Core",
                    "CoreUObject",
                    "Slate",
                    "SlateCore",
                    "Engine",
                    "UnrealEd",
                    "PropertyEditor",
                    "BlueprintGraph",
                    "PointCloudRenderer"
                }
            );

        PrivateDependencyModuleNames.AddRange(
            new string[]
            {
                    "EditorStyle",
                    "AssetRegistry"
            }
        );
    }
}

You can now create a new class (I’d recommend to do it from within Unreal) in your PointCloudRendererEditor module that will contain the actual custom component implementation (I named it „PointCloudRendererComponent“). As stated above we can now call our plugin’s functions like this:

#include "PointCloudRendererComponent.h"
#include "IPointCloudRenderer.h"

// Called when the game starts
void UPointCloudRendererComponent::BeginPlay()
{
	Super::BeginPlay();

	// Connection to the PointCloudRenderer module -> call load function
	if (IPointCloudRenderer::IsAvailable())
		int x = IPointCloudRenderer::Get().LoadPCDFile();
}

Important is now the corresponding header file – in order to show up in the Editor, the class has to inherit from UActorComponent, has to be a UCLASS and the variables have to have the UPROPERTY flag/macro (in our case: filePath):

#pragma once

#include "CoreMinimal.h"
#include "Components/ActorComponent.h"
#include "PointCloudRendererComponent.generated.h"


UCLASS( ClassGroup=Rendering, meta=(BlueprintSpawnableComponent), hideCategories = (Object, LOD, Physics, Collision))
class POINTCLOUDRENDEREREDITOR_API UPointCloudRendererComponent : public UActorComponent
{
	GENERATED_BODY()

public:	
	// Sets default values for this component's properties
	UPointCloudRendererComponent();

	UPROPERTY(EditAnywhere, BlueprintReadWrite, Category = "PointCloudRenderer")
	FString filePath;

protected:
	// Called when the game starts
	virtual void BeginPlay() override;

public:	
	// Called every frame
	virtual void TickComponent(float DeltaTime, ELevelTick TickType, FActorComponentTickFunction* ThisTickFunction) override;

};

This, finally, brings the functionality now to the Editor:


Building PCL / Where to get the right .lib/.dll files from?:

To say this right from the start: Finding and including the right PCL .dll/.lib-files was by far the hardest part of the whole plugin creation. The documentation about this is really bad and I needed weeks to get it working properly. Since I finally included PCL succesfully, I’ll now let you know what I’ve learned.

The most important thing first: I wasn’t able to include PCL as a Dynamic Library (.dll) into Unreal. Maybe there was something wrong with the DLL, but it gave me access violation errors (see Trouble Shooting section). In the end, I included PCL as static libraries (.lib) and didn’t use DLLs at all.

So, in order for you to include your libraries into Unreal, you either find pre-built libraries in the internet (they have to be built in Release mode and with the right Visual Studio compiler version you use) or you build them yourself. I recommend you to build them yourself. I used CMAKE for that. As stated above, I only built static libraries (no DLLs).

For building the libraries, see the official instructions/tutorials on the corresponding official homepages. However, this page might be helpful as well:
http://vadim.macagon.com/blog/2014/10/29/build-a-ue4-compatible-static-lib-in-vs2013/
If you still want to try your luck including PCL DLLs, you can find the newest PCL all-in-one Installers here:
https://github.com/PointCloudLibrary/pcl/releases
For including DLLs however, I’d rather recommend to use Unreal’s Built-in wizard (Plugins –> New Plugin –> Third Party Plugin) since this involves a lot of special code for the DLL handling. Furthermore this might be helpful as well:
https://wiki.unrealengine.com/Linking_Dlls

Trouble shooting:

When you run into problems (which I did a lot), these information might help you:

  • Errors including „#pragma pack(pop)“ / C4103 error: make sure that you do the following in your Build.cs:
    • Enable RTTI with bUseRTTI = true (mandatory for the boost libraries)
    • Enable exception handling with bEnableExceptions = true
    • Add definitions for disabling the ABI headers: Definitions.Add(„BOOST_DISABLE_ABI_HEADERS=1“)
  • Errors including Access Violation, such as:

 

  • This is either a sign that you’re using a Debug version of your DLLs (especially with the _ITERATOR_DEBUG_LEVEL errors) (Unreal uses Release Runtimes only), so you have to build/install them again in Release mode (or maybe try the Github version of Unreal). As stated above, I don’t recommend to use DLLs with Unreal but rather static LIBs.
  • Or that there is a memory/heap corruption between the external library and Unreal or that certain objects are getting garbage collected by Unreal which then probably produce/delete nullpointers. I don’t know how to solve this kind of issues and I don’t recommend to use DLLs with Unreal but rather static LIBs, as stated above.
Blog Comments

Hi Ive been trying to get PCL working with UE4 as well. I got everything going in my plugin and Im at the point of trying to run the sample code from here http://pointclouds.org/documentation/tutorials/greedy_projection.php as soon as I declare the pcl::search::KdTree tree variable it crashes with an access violation. I dont even get a chance to use it, just declare the new tree and it crashes. Ive probably seen all your unreal forum posts when you were trying to get this going and I recall you saying everything was working except for some filter. So I was wondering if you could confirm for me that you can run this example code in unreal, specifically declaring and using a pcl::search::KDTree::Ptr variable. I did try to run the example code outside of unreal and it crashed there to at the same line. Im using PCL 1.8.1 and the latest of the 3rd party programs. Hoping you can help me out as PCL mailing list sucks really bad with pretty much ZERO help. Im not sure if its a PCL code issue or if its my enviroment setup. Thank you for your time and your tutorial in getting me this far.

Hey Chris!
Sorry, I am quite busy, so I didn’t test the whole greedy projection code you mentioned, but I tested the pcl::search::KDTree::Ptr variable successfully. For me, the declaration works and also a quick Nearest Neighbour Search worked as aspected. How are you including the PCL libraries? For me, dynamic libraries (DLLs) didn’t work, which is why I ended up using static libraries (.lib) which I included with e.g. PublicAdditionalLibraries.Add(Path.Combine(ModulePath, „PCL_181/lib/pcl_search_release.lib“));
I would recommend to do the same. If you still have problems, please just write me an email.
Cheers, Valle

[…] A tutorial […]

Leave a Reply to UE4 + VS2017 – The AI Project Cancel Reply

*Please complete all fields correctly

Related Blogs