TutorialMy perspective on converting world coordinates to screen coordinates

Newbie
Meme Tier VIP
Game Name
Any
Anticheat
N/A
N/A
How long you been coding/hacking?
~1.8yrs
Coding Language
C/C++

Your game might use a different matrix orientation. It will be either column major or row major.
Knowledge of matrix multiplication is required.

Hello, recently I've made my first ESP and I went from knowing nothing about the view matrix to knowing enough for an ESP. I'm gonna describe the process.

I learned a lot from these two tutorials:
Rake's tutorial on how to find the view matrix
Mambda's tutorial on the view matrix and ws2()

If you want a geometry perspective on matrix multiplication I highly recommend this video:

Goal: to make a function that you input a world coordinate(3D) and it outputs the corresponding screen coordinate(2D). Let's call such function w2s().

I'll not tell you what the view matrix is or describes because it's not required for an ESP. Also, I don't really know .

w2s() has 2 steps:
1. Multiply the world coordinate vector with the view matrix.
2. Do some scaling to the result of step 1 so it fits nicely on the display.
Step 1
Consider the 3D world coordinate a 4D vector. How? Like this: (x, y, z) -> (x, y, z, 1).
Multiply the 1x4 world position matrix with the 4x4 view matrix. Note: A 3D vector can be viewed like a 1x3 matrix, just like a 4D vector can be viewed like a 1x4 matrix.
The output of this step is a 1x4 matrix(4D vector). Let's call it the screen vector.

Step 2(The scaling step might be different between games, so if this doesn't work print the values to the screen and determine the right way to scale)
screen[0] will eventually be the x screen position(after scaling).
screen[1] will eventually be the y screen position(after scaling).
screen[2] is ignored because a monitor doesn't have depth.
screen[3] is an important value. To be honest I don't know what it is exactly but here's some intuition: screen[0] and screen[1] will vary quite a bit if you change your position/angles. When you divide screen[0] / screen[3] the result should be a value between [-1, 1](the same applies for screen[1]). So think of it as a scaling factor that constrains screen[0] and screen[1] to the range [-1, 1].

Ok, now screen[0] and screen[1] are values in the range [-1, 1](after dividing both by screen[3]). We should convert the range to [0, 1] because it'll be useful later on. You don't need any fancy math for that. Think of an equation that converts a value between these ranges, otherwise look at the spoiler tag .
C++:
``````// Pick any
x/2 + 0.5
0.5*x + 0.5
0.5*(x + 1)``````

The final step is to multiply screen[0] with the width of the game's window and screen[1] with the height of the game's window. That's it, now you have a screen coordinate.

Potential problem
You may ask yourself: What if I apply w2s() to a world coordinate behind me(not in FOV)? The answer is: You'll get garbage values.
The solution is simple: After doing the matrix multiplication check if screen[3] is a value less than 0, if it is the world position is not in your FOV.

Finding the view matrix
As you just saw the view matrix is a matrix of order 4(4x4 matrix). In memory it's probably an array of 16 floats so it's easily recognizable.
Where do you start from? Well, I use Rake's method(at least that's where I saw it first) of searching for +1.0 when looking up and -1.0 when looking down(some games may not allow you to fully look up/down so the value might be 0.98 or something). This will get you down to a few values. Then look at a different angle and delete values which are either +1 or -1 because it should have been somewhere in between. If there're more than a few values try respawning or changing the level with the hope that some values will point to invalid memory. Now you must check value by value to determine which ones are potential good matrices.

Is this a good matrix?

I can guarantee you this matrix will not work, not because it has big values(this might actually be an issue if the values are really big, not in this case though) but because when you do the matrix multiplication 'w' will always be 1. Why? Because we're doing the following to calculate 'w':
C++:
``float w = world_pos[0] * view_matrix[0 * 4 + 3] + world_pos[1] * view_matrix[1 * 4 + 3] + world_pos[2] * view_matrix[2 * 4 + 3] + view_matrix[3 * 4 + 3];``
which will result in:
C++:
``float w = world_pos[0] * 0 + world_pos[1] * 0 + world_pos[2] * 0 + view_matrix[3 * 4 + 3];``
You see... 'w' will always be 1. This cannot happen because we divide x and y by w to scale them to the range [-1, +1] and if 'w' is always 1 it'll not scale.

So, eliminate all values which look like this, then individually test all remaining matrices(there should only be a few)

Possible w2s() function implementation(You just learned how to build your own so this is just to give you an idea):
C++:
``````bool w2s(float* world_pos, float* screen_pos)
{
float* view_matrix = (float*)(base + view_matrix_offset);

float x = world_pos[0] * view_matrix[0 * 4 + 0] + world_pos[1] * view_matrix[1 * 4 + 0] + world_pos[2] * view_matrix[2 * 4 + 0] + view_matrix[3 * 4 + 0];
float y = world_pos[0] * view_matrix[0 * 4 + 1] + world_pos[1] * view_matrix[1 * 4 + 1] + world_pos[2] * view_matrix[2 * 4 + 1] + view_matrix[3 * 4 + 1];
float w = world_pos[0] * view_matrix[0 * 4 + 3] + world_pos[1] * view_matrix[1 * 4 + 3] + world_pos[2] * view_matrix[2 * 4 + 3] + view_matrix[3 * 4 + 3];

if (w < 0.0f)
return false;

x /= w;
y /= w;

screen_pos[0] = (x + 1) * 0.5f * viewport.Width;
screen_pos[1] = (y + 1) * 0.5f * viewport.Height;

return true;
}``````

Attachments

• 2.2 MB Views: 91
Last edited:

XdarionX

Dying Light Hacker
Dank Tier VIP
Dank Tier Donator
Great job! at least game guru has some purpose

Attention! Before you post:

99% of questions are answered in the Beginner's Guide, do it before asking a question.

No Hack Requests. Post in the correct section.  Search the forum first. Read the rules.

How to make a good post:

• Fill out the form correctly
• Tell us the game name & coding language