Talking to your unmanaged code efficiently from Unity using P/Invoke
Sometimes Unity projects need to interface with native plugins that are written in other languages. Other times your company may have a library in another language that you would like to reuse. How can you do that?
Well, if you’re lucky and the library you’re trying to access is written in Java on Android, you can interface with it using the AndroidJNIModule.
In almost all other cases, however, you’ll have to interface your managed C# code with the unmanaged code using P/Invoke, so let’s see what it’s all about!
Table of contents
Unity, Mono, IL2CPP, managed and unmanaged code
The first thing to understand when trying to interface with native plugins is the execution context in which each part of the code runs. To do this we need to look at managed and unmanaged contexts.
Execution contexts: managed vs. unmanaged
In traditional .NET, C# code running inside the .NET virtual machine (VM) is called managed code. Everything running outside of that context is called unmanaged, since it's not managed by the VM.
The VM is not responsible for allocating or deallocating resources — such as memory — or interacting with the outside environment on behalf of the unmanaged code.
Unity, however, does not use the Microsoft .NET runtime environment itself, so let’s see what context the managed code runs in instead.
Unity, Mono, IL2CPP
Historically .NET was only available on Windows, so in order to be cross-platform Unity had to use the Mono runtime instead.
Mono is an open-source project that aims at replicating the runtime environment of .NET for other platforms, such as Linux, OSX, Android, iOS, game consoles, and more. Thus, managed code running inside Mono might as well be running inside .NET as far as it is concerned.
In recent years, though, the Unity developers have started moving from Mono to IL2CPP -a build-time transformation of the C# IL into C++ code- for their runtime environments. This is all fine and good, but does it mean that the code has become unmanaged since it’s now technically C++ code?
In short: no, it’s still managed.
The technical reason why is that IL2CPP actually ships a runtime library which provides the same services as Mono would — memory management, language interoperability, assembly loading, and so on.
So to summarize, no matter whether our C# code was running in a .NET, a Mono or an IL2CPP environment, it would still be the managed side while any plugin it talks to would be the unmanaged side.
Marshalling in Mono/IL2Cpp with P/Invoke
Now that we understand in what context each part of the code runs, how do we make them talk to each other?
Well through P/Invoke, of course!
Wait, what is “P/Invoke”?
One of the services offered by the .NET environments — meaning Mono and IL2CPP as well — is called P/Invoke, short for Platform Invoke.
Its role is to provide mechanisms for managed and unmanaged code to call functions residing on one side from the other, and to marshall data between the two, as parameters or as return values.
Function calls go can either way, and P/Invoke serves as intermediary
The Mono project has a very interesting and extensive document describing P/Invoke and its inner workings. We'll cover the basics here.
Calling a C function from C#
The easiest way to start in our case would be to tell the C# code in Unity about the existence of a C function and how to call it. Let’s assume we are on a POSIX system and take getpid() from the C standard library as an example:
1using System.Runtime.InteropServices;
2
3public static class ProcessContext {
4 [DllImport("libc.so")]
5 public static extern int getpid();
6}Calling the POSIX getpid() function from C#
Let’s break it down!
The first thing that we notice is the DllImport attribute.
- It tells the VM that this function can be found in an unmanaged library.
- The parameter specifies the name of the library in which the function is defined, in our case
libc.so.
This attribute works in combination with the static extern modifiers on the function declaration underneath. They tell the C# compiler that it should not give the function an implicit reference to an instance of the class — which is good because it would not know what to do with it — and that its definition will be found outside of the current assembly.
Then the function is declared with the same signature as in the POSIX specs:
- Its name is
getpid - It takes no argument
- It returns an
int
Note that this declaration has to match with the signature of the function on the unmanaged side, otherwise it will either not build at all, not be found when calling the function at runtime, or cause some other undefined behavior just to ruin your day.
Calling functions written in C++ or other languages
Unfortunately, P/Invoke — outside of the Microsoft environment — is only able to call C ABI-compatible functions in the external library.
In order to use the calling conventions described by the C ABI in C++, you need to put your function definitions inside an extern "C" block like so:
1// Can't be called directly through P/Invoke
2int cpp_only_function(int value);
3
4extern "C" {
5// Everything inside this block can be called through P/Invoke
6int pinvoke_compatible_function(int value);
7
8int here_is_another_one();
9}
10
11// Or they can be declared extern "C" individually
12extern "C" void yet_another_one(double value);Declaring functions as being callable with the C ABI
If however, you must call into libraries written in other languages, either try to find pre-existing interop libraries for C# — which are very likely to be wrapping the target API in C functions — or create those C-style wrappers yourself.
The AndroidJNIModule mentioned in the intro is an example of Unity wrapping the JNI C interface used to interact with the JVM into ready-to-use C# classes.
These were fairly simple examples, but how does argument passing work, and how do we deal with more complex data types?
Marshalling data
When crossing a language barrier, something needs to happen in order to translate the memory representation of data from what is used by the source language to what is expected by the target language.
P/Invoke deals with this by either copying data or even performing full-blown conversions from a source representation to the target one.
This, in essence, is marshalling.
Blittable types
The easiest case is with what are called blittable types. A type is blittable if it has the same memory representation in C# as in the target language.
The upside of blittable types is that their value is directly copied from the managed environment to the unmanaged one without requiring any transformation.
For example, an int in C# becomes int32_t in C, long in C# becomes int64_t , and float becomes float — shocking, I know.
However, the reverse might not be true! C has a great habit of changing the size of its fundamental types depending on the OS, compiler, and processor architecture — yes we’re talking about you, long.
When C’s long is 32 bits wide, in C# it translates to int , but when it’s 64 bits wide, then in C# it translates to long again!
This means it’s a good idea to use explicitly-sized integer types — such as int32_t or uint64_t — on the unmanaged side to avoid sneaky surprises.
Here is a full table of integer conversions in Mono.
The weird case of bool
This brings us to the fundamental type bool in C++.
It is usually 1 byte large, but its size is actually implementation-defined, whereas in C# it always takes 4 bytes.
This discrepancy between the two languages and lack of portability means you should avoid using it on the C++ side of the interface, or explicitly cast it into int32_t first.
Enums
Enums are implicitly converted to their integer representation — int in C# — so they’re also blittable.
However this means that when passing an enum between managed and unmanaged, you need to be sure that their declarations match exactly, in number, order and integer values of the elements, otherwise you’ll have a silent mismatch and you’ll be in for a “fun” debugging session.
1#include <cstddef>
2
3enum class WidgetState : int32_t {
4 On, // => 0
5 State_Missing_In_CSharp, // => 1
6 Off, // => 2
7};
8
9extern "C" int transitionWidgetToState(WidgetState newState);Definition in C++ that has one more state than in C#
1public enum WidgetState
2{
3 On, // => 0
4 Off, // => 1
5}
6
7public class UnmanagedWidget {
8 [DllImport("libwidget.so")]
9 public static extern bool transitionWidgetToState(WidgetState newState);
10
11 public bool TurnOn() => transitionWidgetToState(WidgetState.On); // 0 == 0, Ok
12 public bool TurnOff() => transitionWidgetToState(WidgetState.Off); // 1 != 2, Fails silently!!!
13}Enum definition that doesn’t match
Here you can see that the C++ enum declaration has one more value than the C# one, and no explicit integer value assigned to the different enum values. This is a signal that there is a mismatch, and TurnOff actually sets the state to State_Missing_In_CSharp instead of Off .
Strings
Since the .NET environment comes from Microsoft and was meant to interact with Windows, it considers strings to be encoded in UTF-16, which uses 16-bit characters.
C and C++ however aren’t as prescriptive, and it is very likely that strings would end up being encoded as ASCII or UTF-8, which are 8-bit-wide characters — ok, yeah, UTF-8 is quite a lot more complicated than that, but let’s stick with this approximation for the sake of my sanity and your time.
Other mismatches between the two string representations include:
- String allocation and lifetime management,
- In C#
System.stringis a library type, whereas C considers strings to be null-terminated arrays ofchar— definitely not the same type! - Probably more because dealing with strings in the general case is hard.
In order to solve these issues and make it feasible to pass strings through, the VM automatically copies, converts, and selects the correct type when they cross the interop barrier, assuming they are UTF-8 null-terminated strings on the unmanaged side:
1[DllImport("demo_lib.so")]
2private extern static string returnAString();
3
4[DllImport("demo_lib.so")]
5private extern static void takeAString(string message);C++’s view of the unmanaged functions
This matches the declarations in C++.
1extern "C" {
2const char* returnAString() {
3 return "This string comes from C++!";
4}
5
6void takeAString(const char* message) {
7 std::cout << "This string comes from C#: " << message << "\n";
8}
9}Declaration of the functions taking and returning strings in C++
This conversion step means that strings are not blittable, as they are not simply copied bit-for-bit, and passing them around incurs some conversion cost at runtime.
There is a lot more to be said about the marshalling rules for strings. If you want to dive deeper, refer to the Mono docs.
Structs
C# structs are value types, unlike classes which are reference types. This means their content is copied when passing them as arguments, instead of only passing a reference to the source object. By default, they also have a predictable memory layout, whereas classes don’t.
This means that structs that only contain blittable types are also blittable!
However, as is the case with enums, you must take great care that the definitions between the two languages match exactly.
The absence of a mismatch can’t be checked by the compilers on either side of the interop barrier since they only see one side of the code, and at runtime not enough information is available to perform that check anymore, so blitting structs is totally unchecked!
Another thing to take into account when passing structs is the alignment requirements of each of the fields, or the way the structure has been packed on either side. Once again, if the layouts on both sides don’t match exactly, the marshalling will trigger some undefined behavior with fields not correctly marshalled.
If you want to know more about packing and alignment, you can read Microsoft’s documentation about the topic.
In conclusion
There is more to learn about marshalling between C# and your unmanaged plugins!
For instance, how are exceptions marshalled? Oh they’re not at all and you shouldn’t let them escape through the interop barrier, gotcha.
Additionally, we’ve talked about structs, but how are classes marshalled? Well, by pointer of course (also not forgetting to make even more sure the layout is predictable) .
Can you use [In] or [Out] on parameters? Yes, yes you can.
As you may have understood by now, interop is a broad topic with subtle intricacies and we’ve only scratched the surface here, but if you steer clear of non-blittable types as much as possible and stick with pass-by-value semantics with well-defined type layouts, you can already achieve quite a lot!
An alternative to writing this type of code by hand is to generate the bindings automatically using a tool like SWIG, which still uses P/Invoke behind the scenes and makes sure everything is consistent on both sides.
I’ve been referencing Mono’s documentation about P/Invoke quite extensively throughout this article because it is an excellent resource to try to delve into this topic, so definitely check it out!
Obviously, Microsoft’s documents about P/Invoke are the sources of truth, so if you want to dive even deeper, they are great resources too!
