Documentation Index
Fetch the complete documentation index at: https://mintlify.com/VrajPatel105/cpp-gpu-inference/llms.txt
Use this file to discover all available pages before exploring further.
Deep-copying a large array is expensive. In an ML inference pipeline, a single weight matrix can be hundreds of megabytes — copying it to pass it into a function means allocating that memory a second time, copying every byte, and eventually freeing the duplicate. Move semantics, introduced in C++11, solve this: instead of duplicating a resource, you transfer ownership of it. The original loses access; the destination gains it. No allocation, no copy, O(1) cost regardless of the object’s size. This is not just a performance trick — it is the mental model behind how PyTorch tensors, Eigen matrices, and CUDA buffers are passed around in production ML code.
Lvalue vs Rvalue
Every C++ expression is either an lvalue or an rvalue.
- An lvalue has a persistent memory location that survives past the current expression. You can take its address with
&. Named variables are lvalues.
- An rvalue is a temporary value with no persistent location. It exists only for the duration of the expression that produces it, and is then discarded.
int x = 10; // x is an lvalue — it has a named location in memory
// 10 is an rvalue — it's a temporary integer literal
int y = x + 5; // (x + 5) is an rvalue — a temporary produced by the addition
// y is an lvalue — it names the result's storage location
References and Which Side They Bind To
int& lr = x; // lvalue reference — binds to x (an lvalue) ✓
// int& r = 5; // error — cannot bind lvalue reference to rvalue 5 ✗
int&& rr = 5; // rvalue reference — binds to the temporary 5 ✓
T&&) is what allows a move constructor to receive a temporary and steal its resources.
The Copy Problem
The String class in move_semantics.cpp demonstrates why deep copies are costly. The constructor heap-allocates a char array and copies the string data into it. Every copy of a String triggers another heap allocation and a full memcpy.
#include <iostream>
using namespace std;
class String {
public:
String() = default;
String(const char* string)
{
printf("Created \n");
m_Size = strlen(string);
m_Data = new char[m_Size];
memcpy(m_Data, string, m_Size);
}
~String()
{
delete m_Data;
}
private:
char* m_Data;
uint32_t m_Size;
};
class Entity {
public:
Entity(const String& name)
: m_Name(name) // this copies name — allocates a second char array
{}
private:
String m_Name;
};
int main(){
Entity entity(String("Vraj"));
// String("Vraj") is a temporary rvalue, yet it gets deep-copied into Entity
return 0;
}
String("Vraj") is created, then its contents are copied into Entity::m_Name. The temporary is then immediately destroyed. Two heap allocations were made for one logical string.
std::move and the Move Constructor
std::move does not move anything by itself — it is a cast that converts an lvalue expression into an rvalue reference, signaling to the compiler: “treat this as a temporary; it is safe to steal its resources.”
The actual stealing happens in the move constructor, which accepts T&&:
class MyString {
char* data;
public:
// regular constructor
MyString(const char* s){
data = new char[strlen(s) + 1];
strcpy(data, s);
cout << "Constructed" << endl;
}
// copy constructor — makes a full deep copy
MyString(const MyString& other){
data = new char[strlen(other.data) + 1];
strcpy(data, other.data);
cout << "Copied " << endl;
}
// move constructor — steals the resource, no allocation
MyString(MyString&& other){
data = other.data; // take other's pointer
other.data = nullptr; // leave other in a valid but empty state
cout << "Moved" << endl;
}
void print() {
if (data) cout << data << endl;
else cout << "Empty" << endl;
}
~MyString(){
delete[] data;
cout << "Destroyed" << endl;
}
};
int main(){
MyString s1("Hello");
MyString s2 = move(s1); // s1 cast to rvalue — move constructor fires
s1.print(); // Empty — s1 now holds nullptr
s2.print(); // Hello — s2 owns the original allocation
}
Constructed
Moved
Empty
Hello
Destroyed
Destroyed
How std::move Works
// Simplified — std::move is just a static_cast
template<typename T>
typename std::remove_reference<T>::type&& move(T&& t) noexcept {
return static_cast<typename std::remove_reference<T>::type&&>(t);
}
Cast with std::move
move(s1) produces a MyString&& — an rvalue reference to s1’s memory.
Move constructor is selected
The compiler finds MyString(MyString&& other) and calls it with s1’s data.
Resources are stolen
data = other.data transfers the pointer. other.data = nullptr ensures
s1’s destructor does nothing harmful.
Both destructors run safely
s2’s destructor frees the allocation. s1’s destructor calls
delete[] nullptr, which is a no-op.
Lvalue vs Rvalue: Summary
| Property | Lvalue | Rvalue |
|---|
| Has a name | Yes | No (or treated as temporary) |
| Persistent address | Yes | No |
Can take & address | Yes | No |
Bound by T& | Yes | No |
Bound by T&& | No | Yes |
After std::move | Treated as rvalue | — |
In ML code, move semantics apply directly to tensor buffers. When returning a
large vector<float> weight matrix from a loader function, return it by
value — the compiler will apply NRVO (Named Return Value Optimization) or
the move constructor automatically. Explicitly writing return std::move(mat)
can actually prevent NRVO, so prefer plain return mat for local variables.
Reserve explicit std::move for cases where you are passing an lvalue into a
function or container that you know you will no longer use.