Introduction

Back when I had a 2003 Honda Element EX 4WD 5MT, I had always wanted to mess with my ECU. A lot of my friends had cars that were really good platforms for tuning and watching them plug in a computer, look at a bunch of numbers and crazy-looking graphs, and somehow magically make their car faster made me really jealous. Unfortunately, the K24 in my Element had an ECU that was not possible to reflash without buying some expensive hardware, and even if I did manage to reflash it, there aren’t really that many knobs I could turn that would gain me anything. As such, I didn’t really think about tuning it until the Earth reclaimed my pinch welds and the car fell apart.

My new car

Despite being the world’s biggest E9X hater (I actually have years of receipts for this), I somehow ended up with a manual E92 328i xDrive M Sport as my next car. It’s not something I ever imagined myself in, but it was a deal that was too good to pass up and it was in really good condition (aside from the mysterious wheel speed fault that the dealer wanted several thousand dollars to fix and I in my infinite hubris assumed I could fix for cheap (good thing I was right)).

I was like a kid in a candy shop learning about all the electronic systems in this car. Everything has its own computer. EVERYTHING is on CAN bus. Coming from my Element, this car felt like a spaceship. It doesn’t have a dipstick! It doesn’t use a throttle body! You need a computer with proprietary software not available to the end-user to change the battery! Even better, there actually existed tools to mess with it all. Most of the modules had parameters that, if you knew what you were doing, you could change to make the car behave differently. I could plug in my ThinkPad and change the number of blinks the turn signals did when I tapped the turn signals, or how the brakes behaved, or the state machine for the sunroof. None of this software was supposed to be available to the end-user and as such most of it is in German and looks like it was made in an afternoon by an intern one day in 1997, and that only fueled my drive to learn more. There was no end to the things I could learn about and play with.

And… I could reflash my ECU.

The problem with tuning my car

Thanks to Bimmerlabs, I had a complete ROM dump for the ECU (which I will henceforth also refer to as the DME for BMW reasons) calibration, and the tools to defeat the checksum verification that only allows it to boot things that were signed by BMW. Also, I happened to be good friends with redblockpowered, who is an expert in tuning old Volvos and Saabs and was also interested in learning this new platform.

Typically, to mess with the tuning of a car, you flash new data to the ECU’s ROM that give it different maps that describe the ignition timing, fuelling, etc. To create these modified maps, you typically load a ROM dump into a piece of software such as TunerPro that lets you modify these tables given a definition file that describes the layout of the ECU’s ROM. The definition file tells you the addresses of each table or scalar, as well as how its axes or contents are defined. For example, here is the TunerPro definition for the KF_HUBE table (which controls intake valve lift) on a different DME:

  <XDFTABLE uniqueid="0xe37" flags="0x0">
    <title>KF_HUBE (Sollwert Einlassventilhub UGD)</title>
    <description>Sollwert Einlassventilhub UGD</description>
    <CATEGORYMEM index="0" category="0" />
    <CATEGORYMEM index="1" category="794" />
    <CATEGORYMEM index="2" category="1" />
    <XDFAXIS id="x" uniqueid="0x0">
      <EMBEDDEDDATA mmedtypeflags="0x01" mmedaddress="0x5C548" mmedelementsizebits="16"
        mmedcolcount="20" mmedmajorstridebits="16" mmedminorstridebits="0" />
      <units>RPM</units>
      <indexcount>20</indexcount>
      <decimalpl>0</decimalpl>
      <embedinfo type="1" />
      <datatype>0</datatype>
      <unittype>0</unittype>
      <DALINK index="0" />
      <MATH equation="(X*1)+0">
        <VAR id="X" />
      </MATH>
    </XDFAXIS>
    <XDFAXIS id="y" uniqueid="0x0">
      <EMBEDDEDDATA mmedtypeflags="0x01" mmedaddress="0x5C570" mmedelementsizebits="16"
        mmedcolcount="20" mmedmajorstridebits="16" mmedminorstridebits="0" />
      <units>%</units>
      <indexcount>20</indexcount>
      <decimalpl>4</decimalpl>
      <embedinfo type="1" />
      <datatype>0</datatype>
      <unittype>0</unittype>
      <DALINK index="0" />
      <MATH equation="(X*0.01)+0">
        <VAR id="X" />
      </MATH>
    </XDFAXIS>
    <XDFAXIS id="z">
      <EMBEDDEDDATA mmedtypeflags="0x04" mmedaddress="0x5C598" mmedelementsizebits="16"
        mmedrowcount="20" mmedcolcount="20" mmedmajorstridebits="0" mmedminorstridebits="0" />
      <units>mm</units>
      <decimalpl>3</decimalpl>
      <min>0.000000</min>
      <max>255.000000</max>
      <outputtype>1</outputtype>
      <MATH equation="(X*0.001)+0.">
        <VAR id="X" />
      </MATH>
    </XDFAXIS>
  </XDFTABLE>

This definition tells us that at address 0x5C598 there is a 20x20 table of 16-bit values. The mmedtypeflags field also informs us that this data is signed, and has the least significant bit first (little-endian). Using the units in the x and y axes, we can modify this table to do whatever we want.

Remember how I said this example was from a different DME? Well, that is because it turns out that there is no publicly available definition file for my particular DME, the MSV80.1. You see, the E92 328i was sold with 2 different engines depending on what state you bought the car in — either the N51 or N52. Both engines make effectively the same power, with the N51 being a SULEV variant of the N52 that has a lower compression ratio and more emissions equipment, but a fancy variable-length intake manifold to compensate. They also have different DMEs, with the N52 having the MSV70 and the N51 having the MSV80.1. Of course, there is a definition file floating around the internet for the MSV70, but not the MSV80.1.

Without a definition file I am flying blind through 2 megabytes of random numbers trying to figure out which ones make my car faster.

Reverse engineering the MSV80.1

Aftermarket tunes do exist for the N51/MSV80.1. People will sell you these tunes, but these are likely the result of many many many hours of very smart people with lots of domain-specific knowledge painstakingly reverse-engineering ROM dumps. One of these tunes usually costs in the ballpark of $500. As such, there’s no way they would give you their definition files. If we want one, it seems like we’re on our own.

Thanks to the hard work of hassmaschine, the creator of Bimmerlabs, we know about some architectural differences between the MSV70, and MSV80.1. The MSV70 ROM is big-endian, and the MSV80.1 is little-endian. Notably, much of the calibration data is actually exactly the same. Thus, if we do some transformations to the data such that they are in a common format, we could construct a search algorithm that finds the locations of matching tables in the MSV80.1 ROM, and use these to construct a definition file.

I’m not the first person to think of this — hassmaschine and several others have described working towards a similar algorithm. However, none of this code or its results were publicly available, so I decided to take a stab at creating my own tool. I spent the next few days staying up overnight writing a Python script to do this. A rough description of the algorithm I implemented to find a given table follows:

The Algorithm

• Load definition file and extract addresses, type flags, column/row counts, and other metadata
• Use metadata to extract table from MSV70 ROM, load ROM into a common format
• Calculate offsets for each axis from whichever axis has the lowest axis
• Iterate over each address in the MSV80.1 ROM
• Extract table candidate from MSV80.1 ROM using iterated address and previously calculated offsets (address + offset), load into a common format
  • Compare to known table from MSV70 ROM and calculate distance between the two using some sort of heuristic (I normalized both, subtracted one from the other, and then calculated the matrix norm)
  • Push tuple containing (score, address) to a heap queue so we can later extract the addresses with the top scores
• Extract addresses with top scores for manual review

This algorithm actually worked pretty alright. I was able to find some important tables. A major limitation of this approach is that it’s basically impossible to find scalars since these are only one value long and you are guaranteed to have thousands of exact matches somewhere in the ROM.

I don’t feel that this code is of sufficient quality that I feel good about posting it online. It’s super hacky and more work needs to be done to generalize it beyond my extremely specific task. However, if you think this code would be helpful to you in your own reverse engineering, contact me and I can help you get some code running that helps.

Tuning the N51

Of course, I couldn’t beat redblockpowered at his own game, as he quite literally found most of the important tables bye eye, just scrolling through a hex editor. I did find some important tables, but by the time I was done with my program, he had already finished working on a tune we could try. I cannot understate how good redblockpowered is at this.

I’ve been running a tune made by redblockpowered based on our reverse engineering adventures for a few months now. It feels great. The primary thing that I was interested in tuning is the throttle response, since by default it felt like the throttle was going through a low-pass filter with a cutoff at like 1 Hz. We managed to make the throttle map much more linear, and it feels a lot more responsive and is much easier to downshift smoothly.

Theoretically there should be a bit more power as well, but this is likely outside of what I am able to measure with my tools. I did some datalogging and analysis with Virtual Dyno and although my methodology was as sound as I could make it given the tools available, I found that there was a lot of variance in my runs and I could construct an analysis that would convince me of basically anything.

Future work

• Test power gains on an actual dynamometer
• Generalize the RE algorithm to be usable for other applications

redblockpowered

If you have an old Volvo or Saab, or any other weird one-off tuning project like my N51, I highly recommend you check out redblockpowered.