Pratik Devkota

Detecting Virtualization

2025-02-16T00:00:00+00:00

Background</h2>
A few months ago, I was listening to an episode of Tech Over Tea</a> by Brodie Robertson</a>, featuring Eric Parker</a> as a guest. One part of their discussion that stood out to me was about the feasibility of VM escapes.
What really caught my attention was Eric's explanation of how malware detects whether it's running inside a virtual machine (and basically not even bother if so). Among the various techniques he mentioned, the most interesting one (to me) involved measuring the clock cycles required to execute certain instructions, such as `cpuid</code>. If the execution time is significantly higher than expected (e.g., 500), it's a strong indicator that the code is running in a VM. You can check out that part of the discussionhere</a>.`
`Ever since, I had been meaning to try this myself but kept putting it off, until a few days ago. So in this post, I'll walk through the code I used to test this trick.`

Hardware Virtualization</h2>
At first, you might wonder: wouldn't every instruction be slower in a VM since it runs on top of a
hypervisor and requires emulation? Well, not exactly. Modern processors support virtualization
extensions (e.g., Intel VT-x, AMD-V) that allow the hypervisor to run the guest OS directly on the
hardware, avoiding the need for full emulation. This enables most instructions to execute at
near-native speed.</p>
That was my understanding too: under setups like QEMU+KVM, guest instructions should generally run
as if they were on bare metal. However, as Eric pointed out, some instructions are still emulated. I
have yet to explore all of them, but cpuid</code> stood out as a good candidate to test.</p>

TODO</strong>: how hardware virtualization works. and why cpuid</code> is an exception.</p>
</blockquote>
Measuring Clock Cycles</h2>
How do we measure execution time at such a fine level? In most benchmarks, we'd use a
high-resolution timer provided by the OS. However, OS timers might not be precise enough for
measuring a single instruction's execution. Instead, a better approach I had learned a while back is
to use the Time Stamp Counter (TSC)</strong> provided by the CPU.</p>
The TSC</strong> is a 64-bit register that increments with each clock cycle/tick (1 every CPU Hz)1</a></sup>. On
x86_64 processors, we can read the TSC using the rdtsc</code> instruction. Executing the instruction will
store the current value of the TSC in the edx:eax</code> register pair (lower 32 bits in eax</code> and upper
in edx</code>). So to get the complete 64-bit value, we will need to combine the two registers with some
bitwise operations.</p>
We can use Zig's inline assembly</a> features to
execute the rdtsc</code> instruction, like so:</p>
fn </span>rdtsc</span>() </span>u64 </span>{
</span>    </span>// higher and lower 32-bits
</span>    </span>var </span>high</span>: </span>u32 </span>= 0</span>;
</span>    </span>var </span>low</span>: </span>u32 </span>= 0</span>;
</span>    </span>asm </span>volatile </span>("rdtsc"        </span>// execute the instruction
</span>        : [eax] "={eax}" (low),  </span>// put value of eax in `low`
</span>          [edx] "={edx}" (high), </span>// put value of edx in `high`
</span>    );
</span>    </span>// combine the two values
</span>    </span>return @as</span>(u64, (</span>@as</span>(u64, high) </span><< 32</span>) </span>| </span>(low));
</span>}
</span></code></pre>
We can now use this function to measure the time taken to execute an instruction:</p>
const</span> start </span>= </span>rdtsc</span>();
</span>// execute the instruction
</span>...
</span>// find the delta
</span>const</span> cycles_taken </span>= </span>rdtsc</span>() </span>-</span> start;
</span></code></pre>
Using cpuid</code></h2>
Now, as for the cpuid</code> instruction, Wikipedia has a really good article on it. But basically, its
usage depends on the value of eax</code> before executing it. If eax</code> is 0, it returns the CPU
manufacturer ID string</strong> in ebx</code>, edx</code> and ecx</code> (among other things)2</a></sup>. So we can use this to
identify which processor we are running on.</p>
Let's see an example usage:</p>
const</span> std </span>= @import</span>("std");
</span>pub </span>fn </span>main</span>() </span>void </span>{
</span>    </span>printVendor</span>();
</span>}
</span>fn </span>printVendor</span>() </span>void </span>{
</span>    </span>var </span>ebx</span>: </span>u32 </span>= 0</span>;
</span>    </span>var </span>edx</span>: </span>u32 </span>= 0</span>;
</span>    </span>var </span>ecx</span>: </span>u32 </span>= 0</span>;
</span>    </span>asm </span>volatile </span>(
</span>    </span>// EAX=0: Manufacturer ID
</span>        \\xorl %%eax, %%eax
</span>        </span>// output to EBX, EDX, ECX
</span>        \\cpuid
</span>        : [ebx] "={ebx}" (ebx),
</span>          [edx] "={edx}" (edx),
</span>          [ecx] "={ecx}" (ecx),
</span>        :
</span>        : "eax"
</span>    );
</span>    </span>const </span>vendor</span>: [</span>4</span>]</span>u32 </span>=</span> .{ ebx, edx, ecx, </span>0 </span>};
</span>    </span>const</span> vendorStr </span>= @as</span>([</span>*</span>]</span>const</span> u8, </span>@ptrCast</span>(</span>@alignCast</span>(</span>&</span>vendor)))[0</span>..12</span>];
</span>    std.debug.</span>print</span>("VendorID: {s}</span>\n</span>", .{ vendorStr});
</span>}
</span></code></pre>
Here's what running this code on my machine gives:</p>
$</span> zig run cpuid.zig
</span>VendorID:</span> GenuineIntel
</span></code></pre>
Combining the Two</h2>
So, after combining the knowledge we have gather so far, here's the final code for my TSC check:</p>
const</span> std </span>= @import</span>("std");
</span>
</span>pub </span>fn </span>main</span>() </span>void </span>{
</span>    </span>printVendor</span>();
</span>}
</span>
</span>fn </span>printVendor</span>() </span>void </span>{
</span>    </span>var </span>ebx</span>: </span>u32 </span>= 0</span>;
</span>    </span>var </span>edx</span>: </span>u32 </span>= 0</span>;
</span>    </span>var </span>ecx</span>: </span>u32 </span>= 0</span>;
</span>    </span>const</span> start </span>= </span>rdtsc</span>();
</span>    </span>asm </span>volatile </span>(
</span>        </span>// EAX=0: Manufacturer ID
</span>        \\xorl %%eax, %%eax
</span>        </span>// output to EBX, EDX, ECX
</span>        \\cpuid
</span>        : [ebx] "={ebx}" (ebx),
</span>          [edx] "={edx}" (edx),
</span>          [ecx] "={ecx}" (ecx),
</span>        :
</span>        : "eax"
</span>    );
</span>    </span>const</span> end </span>= </span>rdtsc</span>();
</span>    </span>const </span>vendor</span>: [</span>4</span>]</span>u32 </span>=</span> .{ ebx, edx, ecx, </span>0 </span>};
</span>    </span>const</span> vendorStr </span>= @as</span>([</span>*</span>]</span>const</span> u8, </span>@ptrCast</span>(</span>@alignCast</span>(</span>&</span>vendor)))[0</span>..12</span>];
</span>    std.debug.</span>print</span>("VendorID: {s}, took: {d}</span>\n</span>", .{ vendorStr, end </span>-</span> start });
</span>}
</span>
</span>fn </span>rdtsc</span>() </span>u64 </span>{
</span>    </span>var </span>high</span>: </span>u32 </span>= 0</span>;
</span>    </span>var </span>low</span>: </span>u32 </span>= 0</span>;
</span>    </span>asm </span>volatile </span>("rdtsc"
</span>        : [eax] "={eax}" (low),
</span>          [edx] "={edx}" (high),
</span>    );
</span>    </span>return @as</span>(u64, (</span>@as</span>(u64, high) </span><< 32</span>) </span>| </span>(low));
</span>}
</span></code></pre>
Running this on my Ubuntu VM (top) and my host machine (bottom):</p>
</p>
As expected, the execution time is significantly higher in the VM compared to my bare metal host.</p>
Thanks for reading! I hope you found this as interesting as I did. I plan to dig deeper about this
(for which I am deliberately leaving a TODO for now). So if you're knowledgeable about this topic or
have any insights, feel free to contact me!</p>
References</h3>
^1</sup>Time Stamp Counter</a></p>
</div>
^2</sup>CPUID</a></p>
</div>

Dynamic Memory Allocation: Implementation and Misuse 2024-07-10T00:00:00+00:00 This article has been published at The Medicon Engineering Themes</a>. Find the pdf here</a>. An interesting approach for bytecode dispatch 2023-11-18T00:00:00+00:00 Today I implemented a way to perform bytecode dispatch that I found interesting. It's supposedly called direct threaded code or threaded code interpretation, but my implementation is a lot simpler than how that sounds... What's in a name? Before I get into it, let's first have some context. Following the second part of Robert Nystrom's Crafting Interpreters</a>, I have reached the part where they show how to build a virtual machine</a>. I have my implementation written in Zig, but the implementation is easily application to other languages. There's an enum that is used to determine which instruction is currently being executed: const Opcode = enum(u8) { Constant, Return, }; </code></pre> I have kept it short for brevity, but assume this is all of the opcode supported by our virtual machine. The VM is given a value containing a slice of bytes ([]u8</code>). Some of the bytes in that slice correspond to instructions (opcode and operand) and some is just data. Here's a rough sketch of how the VM is structured: const VM = struct { chunk: []u8, pub fn init(chunk: []u8) VM { return VM { .chunk = chunk, }; } pub fn run(self: *VM) !void { // interpret instructions from `chunk` } }; </code></pre> The VM</code> just stores a slice of bytes which the run()</code> function is supposed to use. The run()</code> function is the actual meat of our VM. It fetches, decodes and executes one bytecode at a time. Typically, we would implement the function like so: const VM = struct { ... fn run(self: *VM) !void { // instruction pointer var ip: usize = 0; while (ip < self.chunk.len) { // fetch a byte and convert it into an instance of `Opcode` const opcode: Opcode = @enumFromInt(self.chunk[ip]); switch (opcode) { .Constant => { // possibly a large block of code }, .Return => { // possibly a large block of code }, } } } }; </code></pre> We use a virtual instruction pointer to keep track of where we are inside the chunk</code>. In a loop, we fetch a byte and use @enumFromInt</code>, a built-in function provided by Zig, that converts an integer into an enum value1</a>. Then follows the dispatcher code. We look up which Opcode</code> variant our opcode</code> represents, and jump to the appropriate label for handling that variant. Since every instruction is handled from that function, that makes it the most performance critical part of our virtual machine. However, this approach has several inefficiencies. First of all, it is difficult for a processor to predict where a jump is going to occur. The switch may translate to a jump that covers most, perhaps even all, of the opcodes. I may have shown only two jump targets, but a real VM would need to handle a lot more than that. There are limits to how many jump targets that the branch predictor</a> can remember. Moreover, the target address calculation consists of instructions being executed at every iteration, which is mostly redundant. Modern processors use instruction caches to drastically improve performance. However, the code to handle each of the opcode variants may be arbitrarily large such that our interpreter loop may not fit into the cache. Such inefficiencies have led programmers to find clever approaches to perform bytecode dispatch. The direct threaded code approach involves creating a buffer holding addresses to all of the jump targets. The buffer is populated once in advance and outside the interpreter loop. Each of the element in the buffer is indexed by the numerical value of the opcode. Thus, the bytecode dispatch corresponds to fetching the opcode and jumping directly to the target indexed by the opcode. For our implementation, I have first defined various functions to handle each of the opcode variants separately. Our VM structure may look something like: const VM = struct { chunk: []u8, pub fn init(chunk: []u8) VM { return VM { .chunk = chunk, }; } pub fn run(self: *VM) !void { // interpret instructions from `chunk` } pub fn handle_return(self: *VM, ip: *usize) void { // code to handle Return } pub fn handle_constant(self: *VM, ip: *usize) void { // code to handle Constant } }; </code></pre> Then we add a field to VM</code> that holds function pointers. For convenience, I have also defined a new type: OpcodeHandler</code>, which is a constant pointer to a function that handles an opcode. Then I add a new field to the structure for storing those OpcodeHandler</code>s. const VM = struct { // a function that handles a particular Opcode variant const OpcodeHandler = *const (fn (self: *VM, ip: *usize) void); chunk: []u8, handlers: []OpcodeHandler ... }; </code></pre> I have updated the initializer slightly so that it accepts an allocator. Now in the initializer, we call a new function get_handlers()</code> that allocates an array and populates it with the required values. I am using a feature provided by Zig that allows us to get the number of variants contained in an enum. We use that length to find the maximum integer value that an Opcode</code> variant might hold. That value should be the length of our handlers</code> array. The implementation of get_handlers()</code> should make this clearer. const VM = struct { const OpcodeHandler = *const (fn (self: *VM, ip: *usize) void); chunk: []u8, handlers: []OpcodeHandler pub fn init(allocator: Allocator, chunk: []u8) VM { return VM { .chunk = chunk, .handlers = get_handlers(allocator), }; } ... fn get_handlers(allocator: Allocator) []OpcodeHandler { // total number of variants in Opcode const total = @typeInfo(Opcode).Enum.fields.len; var handlers = allocator.alloc(OpcodeHandler, total) catch @panic("OOM"); // store the function pointers at the index corresponding to the opcode value for (0..total) |i| { const opcode: Opcode = @enumFromInt(i); switch (opcode) { .Return => { handlers[i] = VM.handle_return; }, .Constant => { handlers[i] = VM.handle_constant; }, } } return handlers; } pub fn handle_return(self: *VM, ip: *usize) void { // code to handle Return } pub fn handle_constant(self: *VM, ip: *usize) void { // code to handle Constant } }; </code></pre> Now the interpreter loop only needs to fetch an opcode and call the function at the index given by the opcode value, instead of doing the switch. const VM = struct { ... fn run(self: *VM) !void { var ip: usize = 0; while (ip < self.chunk.len) { const opcode: Opcode = @enumFromInt(self.chunk[ip]); self.handlers[opcode](self, &ip); } } }; </code></pre> And there you go, a much simpler and more efficient solution! ^{1 errors aren't possible and the sun rises from the west </div>} Generating documentation from zig build 2023-11-16T00:00:00+00:00 Background</h2> So a few days ago, I began following the second part of Robert Nystrom's Crafting Interpreters</a>. The author is doing it in C, but I figured Zig would be the better tool because I get to enjoy some modern features like the build system while also having the low-level power of C. I took this project to also serve as a means to get more familiar with Zig. So, I have been forcing myself to go on whatever tangent ideas that occur to me. One of those ideas was a way to generate documentation on demand. I had done the first part of the book in Rust, so I was very used to cargo</code> and running cargo doc</code> to generate my package's documentation. Zig has the -femit-docs</code> flag (with zig build-exe</code>) that triggers generation of documentation when building an executable. However, I couldn't find a way to do the same with the build system. The default build.zig</code> file created when we run zig init-exe</code> doesn't enable emitting docs. The one place</a> where I found a way to enable it was apparently also outdated. To clarify, what I wanted was something similar to cargo doc</code> where running something like zig build docs</code> would generate the documentation for me. For some reason1</a> I got stuck on this for days although I did stumble across some great</a> articles</a> along the way, and some nice folks on IRC who helped me figure this out. All in all, I feel like I have a better understanding of not just the build system but Zig in general because of this endeavour. I started this blog promising myself that I'd document the things I learned. So, I figured I should write an article about it; summarising this little rabbit hole I just came out of. The build binary</h2> Couldn't I just pass the regular -femit-docs</code> flag to zig build</code>? Apparently not. Executing zig build</code> is actually a two-step process. First, zig</code> builds a binary from the lib/build_runner.zig</a> file. We can see that the file is supposed to be an executable, due to the presence of an entrypoint (pub fn main() !void</code>)2</a>. Reading its contents, we can infer the arguments that the executable expects. Some of those arguments may even seem familiar to that of zig</code> itself, but it's a completely different binary with its own set of arguments. So, this binary combined with the build.zig</code> file that we have defined is what's going to actually build our project. Two steps: first the build binary, then our binary. Steps</h2> The whole build system is modelled as a directed acyclic graph</a> composed of Step</a>s. The build</code> function in our build.zig</code> file is given a pointer to an instance of std.build.Build</code>. Inside the function, we declaratively define a set of Step</code>s that the build binary should execute. The build</code> function doesn't actually execute anything, it only defines the steps and their dependencies. The documentation for Step</code> shows that every step has an id</code> field which corresponds to the type of the step. A top level step is one that can be invoked by a name: zig build <name></code>. Remembering our goal - getting zig build docs</code> to work - we probably should look into creating a top level step. The Install step</h2> Let's first look at what we have. Here's a minimal build.zig</code> that we're starting with: const std = @import("std"); pub fn build(b: *std.Build) void { const exe = b.addExecutable(.{ .name = "myprogram", .root_source_file = .{ .path = "src/main.zig" }, .target = b.standardTargetOptions(.{}), .optimize = b.standardOptimizeOption(.{}), }); b.installArtifact(exe); } </code></pre> Running zig build -l</code> with this file gives us the two default top-level steps: install</code> and uninstall</code>. The purpose of the Install step is to copy build artifacts to a prefix path. A build artifact is any file or directory that is created during the build process which isn't the primary output of the program. This includes things like object files, libraries, and executables. The prefix path is usually just ./zig-out/</code>. Defining a step by itself doesn't do anything. It's the dependencies it has that specify what to install. In the example above, we call the addExecutable</code> function. Here's its function signature: fn addExecutable(b: *Build, options: ExecutableOptions) *Step.Compile </code></pre> It returns a Step.Compile</code> pointer. That function creates a step declaring intent for producing an executable. That pointer (stored as the exe</code> constant) is then passed to installArtifact()</code>: /// This creates a new step and adds it to the dependencies of the top-level install step, using all the default options. fn installArtifact(self: *Build, artifact: *Step.Compile) void </code></pre> It should be clear from the doc comment that the function adds a dependency to the install step. There are functions for installing a file or a directory as well: installFile()</code>, installDirectory()</code>. Creating a step</h2> If you've used zig init-exe</code> before, you should find that the default build.zig</code> file it creates has several additional top-level steps defined. Those steps are defined with the step()</code> function: fn step(self: *Build, name: []const u8, description: []const u8) *Step </code></pre> You should be able to guess by now about how I want to relate that information to getting zig build docs</code>. Let's create ourselves a docs</code> step: pub fn build(b: *std.Build) void { ... const docs_step = b.step("docs", "Copy documentation to prefix path"); } </code></pre> Here's what we have now: ❯ zig build -l install (default) Copy build artifacts to prefix path uninstall Remove build artifacts from prefix path docs Copy documentation to prefix path </code></pre> Again, this step by itself doesn't do anything. So let's dig deeper. Triggering documentation generation</h2> If we pass the --verbose</code> flag to zig build</code>, it prints us the command that the build binary is executing: ❯ zig build --verbose /home/dev/bin/zig build-exe /home/dev/myprogram/src/main.zig --cache-dir /home/dev/myprogram/zig-cache --global-cache-dir /home/dev/.cache/zig --name myprogram --listen=- </code></pre> We now need to find a way to add the -femit-docs</code> flag to that output. Alas, as much as I'd enjoy showing a gradual path to finding these things out, I'll have to eventually come to reality. Like all of what I've just explained, the actual path to learning these things isn't linear at all. It wasn't until someone nicked ifreund</code> on Zig's IRC channel gave me the pointers did I figure this out. Our exe</code> constant is a pointer to Step.Compile</a> which contains a member function: // Returns the path to the generated documentation directory. fn getEmittedDocs(self: *Compile) LazyPath </code></pre> I've copied the comment from the source as-is. This is what made me stuck: I misunderstood the comment to mean that there's already a generated path and calling the function just returns a path to it. However, if we look at the source: pub fn getEmittedDocs(self: *Compile) LazyPath { return self.getEmittedFileGeneric(&self.generated_docs); } fn getEmittedFileGeneric(self: *Compile, output_file: *?*GeneratedFile) LazyPath { if (output_file.*) |g| { return .{ .generated = g }; } const arena = self.step.owner.allocator; const generated_file = arena.create(GeneratedFile) catch @panic("OOM"); generated_file.* = .{ .step = &self.step }; output_file.* = generated_file; return .{ .generated = generated_file }; } </code></pre> Now, I am not comfortable in Zig enough to be certain about all of the code but here's what I can analyze: The self.generated_docs</code> field, being passed to getEmittedFileGeneric()</code>, supposedly contains the final path to the generated documentation.</li> Inside getEmittedFileGeneric()</code>, the first check ensures we don't generate files more than once.</li> The rest of the body creates or obtains a GeneratedFile</a> and returns it as part of a LazyPath</a>. All in all, we can infer that calling exe.getEmittedDocs()</code> actually triggers the documentation generation and gives us the path.</li> </ul> Installing the documentation</h2> We have a path to the documentation, but it won't be in our prefix path. As mentioned above, we need to define directories to be installed to the Install</code> step. Let's update our build.zig</code> file. Now, there are two ways to go about doing that. The first one: pub fn build(b: *std.Build) void { ... b.installDirectory(.{ .source_dir = exe.getEmittedDocs(), .install_dir = .prefix, .install_subdir = "docs", }); } </code></pre> Doing this would add a dependency to the top-level Install</code> step so that the docs are installed every time the Install</code> step is performed. However, we want to generate documentation on demand. Heck, we haven't even used the docs</code> step we created. So, we have to use a more flexible approach. Adding dependency to the docs step</h2> The build system offers this function: fn addInstallDirectory(self: *Build, options: InstallDirectoryOptions) *Step.InstallDir </code></pre> This is similar to installDirectory()</code> but returns us a Step</code> rather than automatically adding the specified directory as a dependency to the Install</code> step. Now we can choose which Step</code> this InstallDir</code> step should depend on. As you may have guessed, we're going to make it depend on our docs</code> step. pub fn build(b: *std.Build) void { ... const install_docs = b.addInstallDirectory(.{ .source_dir = exe.getEmittedDocs(), .install_dir = .prefix, .install_subdir = "docs", }); const docs_step = b.step("docs", "Copy documentation artifacts to prefix path"); docs_step.dependOn(&install_docs.step); } </code></pre> And there you go! Having this as our build.zig</code> gives us the zig build docs</code> command. If we do a zig build --verbose docs</code>, we should see the -femit-docs</code> we wanted: ❯ zig build docs --verbose /home/dev/bin/zig build-exe /home/dev/myprogram/src/main.zig -femit-docs --cache-dir /home/dev/myprogram/zig-cache --global-cache-dir /home/dev/.cache/zig --name myprogram --listen=- </code></pre> For convenience, here's the final version of our build.zig</code>: const std = @import("std"); pub fn build(b: *std.Build) void { const exe = b.addExecutable(.{ .name = "myprogram", .root_source_file = .{ .path = "src/main.zig" }, .target = b.standardTargetOptions(.{}), .optimize = b.standardOptimizeOption(.{}), }); b.installArtifact(exe); const install_docs = b.addInstallDirectory(.{ .source_dir = exe.getEmittedDocs(), .install_dir = .prefix, .install_subdir = "docs", }); const docs_step = b.step("docs", "Copy documentation artifacts to prefix path"); docs_step.dependOn(&install_docs.step); } </code></pre> Thanks for reading! ^{1 skill issue, obviously </div> ^{2 you know C, right? </div>}}

Pratik Devkota

Detecting Virtualization

Combining the Two</h2> So, after combining the knowledge we have gather so far, here's the final code for my TSC check:</p>

References</h3> 1</sup> Time Stamp Counter</a></p> </div> 2</sup> CPUID</a></p> </div>

Dynamic Memory Allocation: Implementation and Misuse

An interesting approach for bytecode dispatch

Generating documentation from zig build

References</h3>
^1</sup>
Time Stamp Counter</a></p> </div>
^2</sup>
CPUID</a></p> </div>