Actions

Game Protocol recap: Difference between revisions

From RuneWiki

No edit summary
No edit summary
 
(6 intermediate revisions by the same user not shown)
Line 1: Line 1:
I'm going to recap everything we know about the game server's protocol so beginners and veterans can refine their understanding a bit. We're going to be using some terms that will conflict with the last 20 years of RSPS, keep an open mind if you're already hard set in your ways ;)
I'm going to recap everything we know about the game server's protocol so beginners and veterans can refine their understanding a bit. We're going to be using some terms and ideas that will conflict with the last 20 years of RSPS, keep an open mind!


We'll cover the basics first and touch on more in-depth details after.
We'll cover the basics first and touch on more in-depth details after.


==== Basics ====
==== Basics ====
===== Data Sizes =====
A byte is the smallest unit of data you can store in memory. You can edit the bits that make up a byte, but you're always working with bytes.
A byte is the smallest unit of data you can store in memory. You can edit the bits that make up a byte, but you're always working with bytes.


Signed numbers means they can have a minus sign. They can represent values from -<data limit> to <data limit>.
Signed numbers means they can have a minus sign. They can represent values from -<nowiki><data range> to <data range>.</nowiki>


Unsigned numbers means they cannot have a minus sign. They can represent values from 0 to <data limit>.
Unsigned numbers means they cannot have a minus sign. They can represent values from 0 to <nowiki><data range></nowiki><data limit>.


Java int types are signed and may represent values between -2,147,483,648 to 2,147,483,647 (-2.147b to 2.147b).
Java int types are signed and may represent values between -2,147,483,648 to 2,147,483,647 (-2.147b to 2.147b).
Line 16: Line 18:
Numbers can be converted between signed/unsigned at any time if you store the result in the right data types.
Numbers can be converted between signed/unsigned at any time if you store the result in the right data types.


TCP is a stream protocol, you can expect to send a byte on one end and the other end will receive that byte in the exact same order (simplification).
===== TCP =====
TCP is a byte stream protocol, you can expect to send a byte on one end and the other end will receive that byte in the exact same order (simplification).


Opcodes means "Operation Codes" which is another way to say, if this byte is received then do this. The first byte(s) of a packet are the opcode. Note that I say byte(s), that distinction is for RS after 2010, we'll revisit the reason later.
===== Opcodes =====
Opcodes means "Operation Codes" which is another way to say, if this byte is received then do this. The first byte(s) of a packet are the opcode. Note that I say byte(s), that distinction is for RS after 2010, which can write 2 bytes if the opcode is greater than 127.


===== Isaac =====
Isaac is the stream cipher that Jagex uses to encrypt the opcode. Every byte that it processes changes the state permanently and both client/server must be doing the same operations in the same order at all times.
Isaac is the stream cipher that Jagex uses to encrypt the opcode. Every byte that it processes changes the state permanently and both client/server must be doing the same operations in the same order at all times.


When the client connects, it initializes Isaac with the exact same starting state as the server.
When the client connects, it initializes Isaac with the exact same starting state as the server.


Packet lengths for these opcodes can be fixed or variable, lengths are important for two reasons: TCP is a byte stream protocol and you need to know how much to read off the socket, and Isaac must be deciphering/decrypting the same opcodes, in the same order, on both sides.
===== Packet Lengths =====
Packet lengths for these opcodes can be fixed or variable, lengths are important for two reasons:
 
TCP has TCP fragmentation, where packets are split between multiple transmissions
 
Isaac must be deciphering/decrypting the same opcodes, in the same order, on both sides.


A fixed packet length is calculated by adding up all of the bytes that are sent and storing that result ahead of time.
A fixed packet length is calculated by adding up all of the bytes that are sent and storing that result ahead of time.
Line 37: Line 47:
When the client is being obfuscated for release, the network protocol gets randomized in a couple ways...
When the client is being obfuscated for release, the network protocol gets randomized in a couple ways...


Opcodes are scrambled (changed randomly each revision and for each platform)
Opcodes are scrambled - changed randomly each revision and for each platform


Reading/writing methods in the packet handlers may be replaced with their alt variants
Reading/writing methods in the packet handlers may be replaced with their alt variants


Packet handlers are shuffled to be in random locations (you cannot simply diff them in the same order after decompiling)
Packet handlers are shuffled around to be in random locations, you cannot simply diff them in the same order after decompiling


Read along here: Packet.java
Read along here: Packet.java


===== Writing =====
===== Writing =====
Jagex Name RSPS Name Description
{| class="wikitable"
p1 writeByte Put 1 byte in the data buffer
|+
p2 writeShort Put 2 bytes in the data buffer
!Jagex Name
ip2
!RSPS Name
Put 2 bytes in the data buffer, inversed (backwards)
!Description
p3 writeMedium Put 3 bytes in the data buffer
|-
p4 writeInt Put 4 bytes in the data buffer
|p1
ip4
|writeByte
Put 4 bytes in the data buffer, inversed (backwards)
|
p5
|-
Put 5 bytes in the data buffer
|p2
p6
|writeWord, writeShort
Put 6 bytes in the data buffer
|
p8 writeLong Put 8 bytes in the data buffer
|-
pjstr writeString Put a Jagex string in the data buffer, NUL or newline terminated
|p3
pjstr2
|writeMedium
Put a Jagex string in the data buffer, with a version byte and NUL or newline terminated
|
pdata writeBytes Put part of another data buffer into the data buffer
|-
psize2
|p4
Put the 0-65535 (2 bytes) size of how many bytes to read ahead into the data buffer
|writeInt
psize1
|
Put the 0-255 (1 byte) size of how many bytes to read ahead into the data buffer
|-
pSmart1or2 / psmart
|p5
Put a "smart" number (1 or 2 bytes) into the data buffer (-16384 to 16383)
|
pSmart1or2s / psmarts
|
Put a "smart" number (1 or 2 bytes) into the data buffer (0 to 32767)
|-
pSmart2or4
|p6
Put a "smart" number (2 or 4 bytes) into the data buffer
|
pBit
|
Put a variable number of bits into the data buffer
|-
|p8
|writeLong
|
|-
|pjstr
|writeString
|
|-
|pjstr2
|
|
|-
|pdata
|writeBytes
|
|-
|psmart
|
|
|-
|psmarts
|
|
|-
|pSmart2or4
|
|
|-
|pBit
|
|
|}


===== Writing Alt (Obfuscation) =====
===== Writing Alt (Obfuscation) =====
Jagex Name RSPS Name Description
{| class="wikitable"
p1_alt1 (todo)
!Jagex Name
p1_alt2
!RSPS Name
 
!Description
p1_alt3
|-
 
|p1_alt1
p2_alt1
|
 
|
p2_alt2
|-
 
|p1_alt2
p2_alt3
|
 
|
p3_alt1
|-
 
|p1_alt3
p3_alt2
|
 
|
p3_alt3
|}
 
p4_alt1
 
p4_alt2
 
p4_alt3
 
pdata_alt1
 
pdata_alt2
 
pdata_alt3


===== Reading =====
===== Reading =====
Jagex Name RSPS Name Description
{| class="wikitable"
g1 readByte Get 1 byte from the data buffer
!Jagex Name
g2 readShort Get 2 bytes from the data buffer
!RSPS Name
g3 readMedium Get 3 bytes from the data buffer
!Description
g4 readInt Get 4 bytes from the data buffer
|-
g5
|g1
Get 5 bytes from the data buffer
|
g6
|
Get 6 bytes from the data buffer
|-
g8 readLong Get 8 bytes from the data buffer
|g2
gjstr readString Get a Jagex string from the data buffer, NUL or newline terminated
|
gjstr2
|
Get a Jagex string from the data buffer, with a version byte and NUL or newline terminated
|-
gdata readBytes Get part of this data buffer and put it into another data buffer
|g3
gSmart1or2 (unofficial) / gsmart (official)
|
Get a "smart" number (1 or 2 bytes) from the data buffer (-16384 to 16383)
|
gSmart1or2s (unofficial) / gsmarts (official)
|-
Get a "smart" number (1 or 2 bytes) from the data buffer (0 to 32768)
|g4
gSmart1or2null (unofficial)
|
Get a "smart" number (1 or 2 bytes) from the data buffer (-1 to 32768) and treat -1 as "null"
|
gExtended1or2 (unofficial)
|-
Get a variable amount of "smart" numbers from the data buffer until there's none left
|g5
gSmart2or4 (official)
|
Get a "smart" number (2 or 4 bytes) from the data buffer
|
gBit
|-
Get a variable number of bits from the data buffer
|g6
|
|
|-
|g8
|
|
|-
|gjstr
|
|
|-
|gjstr2
|
|
|-
|gdata
|
|
|-
|gsmart
|
|
|-
|gsmarts
|
|
|-
|gBit
|
|
|}


===== Reading Alt (Obfuscation) =====
===== Reading Alt (Obfuscation) =====
Jagex Name RSPS Name Description
{| class="wikitable"
g1_alt1 (todo)
!Jagex Name
g1_alt2
!RSPS Name
 
!Description
g1_alt3
|-
 
|g1_alt1
g2_alt1
|
 
|
g2_alt2
|}
 
g2_alt3
 
g3_alt1
 
g3_alt2
 
g3_alt3
 
g4_alt1
 
g4_alt2
 
g4_alt3
 
gdata_alt1
 
gdata_alt2
 
gdata_alt3


==== Server Logic ====
==== Server Logic ====
The server has some unique behaviors when processing packets. RSProt has some info covering packet limits that the server receives but I'll go over that here as well.
The server has some unique behaviors when processing packets. RSProt has some info covering packet limits that the server receives but I'll go over that here as well.


Client packets are separated into two types - client events (tracking/handled internally by the client) and user events (actionable inputs).
Client packets are separated into two types - client events (tracking/handled internally by the client) and user events (actionable inputs).


When the server reaches the limit of 10 user events or 50 client events, packet decoding stops for that player this tick and continues again next tick. These limits may vary depending on your revision if you care about that level of detail, OSRS made a change at one point to increase it to 10 user events from 5.
When the server reaches the limit of 10 user events or 50 client events, packet decoding stops for that player this tick and continues again next tick. These limits may vary depending on your revision if you care about that level of detail, OSRS made a change at one point to increase it to 10 user events from 5.


Server packets are also separated into two types - high priority, and low priority. That's two packet queues you write into, this helps improve game responsiveness because the client will only process 5-100 packets every 20ms. The important packets process first (map/info packets/variables) and the lower-priority content packets appear after.
Server packets are also separated into two types - high priority, and low priority. That's two packet queues you write into, this helps improve game responsiveness because the client will only process 5-100 packets every 20ms. The important packets process first (map/info packets/variables) and the lower-priority content packets appear after.
 
{| class="wikitable"
 
!Priority
 
!Name
Priority Name
|-
High VARP_SMALL / VARP_LARGE
|High
High PLAYER_INFO
|VARP_SMALL / VARP_LARGE
Low IF_SETTEXT
|-
(todo ...) (todo ...)
|High
|PLAYER_INFO
|-
|Low
|IF_SETTEXT
|}


==== Deobfuscating Clients ====
==== Deobfuscating Clients ====


 
I'm going to take you through what it looks like to know how to read a packet.<syntaxhighlight lang="java">
I'm going to take you through what it looks like when you're looking at a correctly named client vs average RSPS client:<syntaxhighlight lang="java">
if(pktType == 176) {
  daysSinceRecovChange = inStream.method427();
  unreadMessages = inStream.method435();
  membersInt = inStream.readUnsignedByte();
  anInt1193 = inStream.method440();
  daysSinceLastLogin = inStream.readUnsignedWord();
  // ...
  pktType = -1;
  return true;
}
</syntaxhighlight><syntaxhighlight lang="java">
if (this.packetType == 176) {
if (this.packetType == 176) {
   this.daysSinceRecoveriesChanged = this.in.g1_alt2();
   this.daysSinceRecoveriesChanged = this.in.g1_alt2();
Line 202: Line 254:
   return true;
   return true;
}
}
</syntaxhighlight><syntaxhighlight lang="java">
</syntaxhighlight>With the right Jagex naming, you can calculate the packet length by just adding 1 (g1) + 2 (g2) + 1 (g1) + 4 (g4) + 2 (g2) and know which methods to use on the "other side" immediately.
if(pktType == 176) {
  daysSinceRecovChange = inStream.method427();
  unreadMessages = inStream.method435();
  membersInt = inStream.readUnsignedByte();
  anInt1193 = inStream.method440();
  daysSinceLastLogin = inStream.readUnsignedWord();
  // ...
  pktType = -1;
  return true;
}
</syntaxhighlight>With the right Jagex naming, you can calculate the packet length by just adding 1 (g1) + 2 (g2) + 1 (g1) + 4 (g4) + 2 (g2) and know which alt methods to read with immediately.


==== Sources ====
==== Sources ====
Lost City - 2004Scape (2024)
Lost City - 2004Scape (2024)
Blurite - RSProt (2024)
Blurite - RSProt (2024)
NXT beta client messages (2016)
NXT beta client messages (2016)
NXT beta server messages (2016)
NXT beta server messages (2016)
NXT beta C++ function names (2016)
NXT beta C++ function names (2016)
Password Applet decompiled (2009)
Password Applet decompiled (2009)

Latest revision as of 15:28, 4 July 2024

I'm going to recap everything we know about the game server's protocol so beginners and veterans can refine their understanding a bit. We're going to be using some terms and ideas that will conflict with the last 20 years of RSPS, keep an open mind!

We'll cover the basics first and touch on more in-depth details after.

Basics

Data Sizes

A byte is the smallest unit of data you can store in memory. You can edit the bits that make up a byte, but you're always working with bytes.

Signed numbers means they can have a minus sign. They can represent values from -<data range> to <data range>.

Unsigned numbers means they cannot have a minus sign. They can represent values from 0 to <data range>.

Java int types are signed and may represent values between -2,147,483,648 to 2,147,483,647 (-2.147b to 2.147b).

Java byte types are signed and may represent values between -128 and 127.

Numbers can be converted between signed/unsigned at any time if you store the result in the right data types.

TCP

TCP is a byte stream protocol, you can expect to send a byte on one end and the other end will receive that byte in the exact same order (simplification).

Opcodes

Opcodes means "Operation Codes" which is another way to say, if this byte is received then do this. The first byte(s) of a packet are the opcode. Note that I say byte(s), that distinction is for RS after 2010, which can write 2 bytes if the opcode is greater than 127.

Isaac

Isaac is the stream cipher that Jagex uses to encrypt the opcode. Every byte that it processes changes the state permanently and both client/server must be doing the same operations in the same order at all times.

When the client connects, it initializes Isaac with the exact same starting state as the server.

Packet Lengths

Packet lengths for these opcodes can be fixed or variable, lengths are important for two reasons:

TCP has TCP fragmentation, where packets are split between multiple transmissions

Isaac must be deciphering/decrypting the same opcodes, in the same order, on both sides.

A fixed packet length is calculated by adding up all of the bytes that are sent and storing that result ahead of time.

A variable packet length is calculated when packets are written, the byte(s) following the opcode in this case are saying how many bytes the rest of the packet is.

Packet Writing/Reading Methods

The client has a class officially called Packet. Yes, even when it's used to read the cache it's still reading as Packet. People often call this Buffer, InputStream/OutputStream, or ByteStream.

Later revisions there's another class called PacketBit, some people have called this Packet. It contains the methods to read/write bits.

When the client is being obfuscated for release, the network protocol gets randomized in a couple ways...

Opcodes are scrambled - changed randomly each revision and for each platform

Reading/writing methods in the packet handlers may be replaced with their alt variants

Packet handlers are shuffled around to be in random locations, you cannot simply diff them in the same order after decompiling

Read along here: Packet.java

Writing
Jagex Name RSPS Name Description
p1 writeByte
p2 writeWord, writeShort
p3 writeMedium
p4 writeInt
p5
p6
p8 writeLong
pjstr writeString
pjstr2
pdata writeBytes
psmart
psmarts
pSmart2or4
pBit
Writing Alt (Obfuscation)
Jagex Name RSPS Name Description
p1_alt1
p1_alt2
p1_alt3
Reading
Jagex Name RSPS Name Description
g1
g2
g3
g4
g5
g6
g8
gjstr
gjstr2
gdata
gsmart
gsmarts
gBit
Reading Alt (Obfuscation)
Jagex Name RSPS Name Description
g1_alt1

Server Logic

The server has some unique behaviors when processing packets. RSProt has some info covering packet limits that the server receives but I'll go over that here as well.

Client packets are separated into two types - client events (tracking/handled internally by the client) and user events (actionable inputs).

When the server reaches the limit of 10 user events or 50 client events, packet decoding stops for that player this tick and continues again next tick. These limits may vary depending on your revision if you care about that level of detail, OSRS made a change at one point to increase it to 10 user events from 5.

Server packets are also separated into two types - high priority, and low priority. That's two packet queues you write into, this helps improve game responsiveness because the client will only process 5-100 packets every 20ms. The important packets process first (map/info packets/variables) and the lower-priority content packets appear after.

Priority Name
High VARP_SMALL / VARP_LARGE
High PLAYER_INFO
Low IF_SETTEXT

Deobfuscating Clients

I'm going to take you through what it looks like to know how to read a packet.

if(pktType == 176) {
   daysSinceRecovChange = inStream.method427();
   unreadMessages = inStream.method435();
   membersInt = inStream.readUnsignedByte();
   anInt1193 = inStream.method440();
   daysSinceLastLogin = inStream.readUnsignedWord();
   // ...
   pktType = -1;
   return true;
}
if (this.packetType == 176) {
   this.daysSinceRecoveriesChanged = this.in.g1_alt2();
   this.unreadMessages = this.in.g2_alt2();
   this.warnMembersInNonMembers = this.in.g1();
   this.lastAddress = this.in.g4_alt3();
   this.daysSinceLastLogin = this.in.g2();
   // ...
   this.packetType = -1;
   return true;
}

With the right Jagex naming, you can calculate the packet length by just adding 1 (g1) + 2 (g2) + 1 (g1) + 4 (g4) + 2 (g2) and know which methods to use on the "other side" immediately.

Sources

Lost City - 2004Scape (2024)

Blurite - RSProt (2024)

NXT beta client messages (2016)

NXT beta server messages (2016)

NXT beta C++ function names (2016)

Password Applet decompiled (2009)