Performance Testing

Performance Testing With Pro Tools

Posted on by

Why Conduct Performance Tests ?

Here at Pro Tools PC we are regularly asked about how the performance of “Computer A” compares to “computer B”. Is it worth it to upgrade?? These questions are obviously difficult to answer due to a huge number of variables. But one way we can try and make things a little easier is by physical performance testing alongside explaining some fundamental technicalities that should always be considered when debating performance spec's of computers but are all too often overlooked. In this article we will attempt to break down this complicated subject matter into sizeable understandable chunks which should in turn help people decide what best fits their needs. If you find yourself asking questions and not coming up with answers please leave us some comments below and we'll be happy to address them or even contact us directly using the "Contact Us" link in the website menu.

Geek Warning!

Some things are very difficult or near impossible to quantify in a generic test. For example, A 2010 Mac pro 8 core or a 4-year-old PC with a 3930k 6 core CPU, may seem fast to someone who has not used anything newer or a more recent system. This may even then, lead them to an over appreciated sense of improvement when compared to something older and thus lead them to believe upgrading to this (now) older tech might be a good investment. However, (perhaps unknown to them) they'll very likely be missing out on important generational developments since that hardware was released often at a near exponential rate such as CPU cache improvements on newer processors and memory bandwidth/clock speed (which are some things that really do make a huge difference). In pure horsepower, a modern 4 core (7700k) has more practical real world power than either of those mentioned. But what is not quantifiable is improvements like the cache as memory built onto the processor for extremely fast retrieval times and instruction sets that a CPU uses for the most important tasks. The more cache, the smoother and more responsive your system will be. While it may not relate to pure “horsepower” it is an important part of the overall experience of a computer. This leads us into the obvious RAM discussions. Starting at the 6th generation of the 4 cores and the 5th generation of the E series processors (6 core and up) the RAM type has changed to DDR4. It is not compatible with the previous DDR3 systems. The ram speed itself has increased and voltage has lowered (resulting in less heat, therefore more efficient). Actual buss bandwidth between the CPU and memory has almost doubled since the first i7 920 processor. Couple this with how the increased clock speed of the CPU x memory multiplier, and it’s easy to see the transfer speed and capabilities of the RAM are greatly improved.

What We Can Measure

In the past, we have always used the D-Verb test that came from the DUC and I believe started by Allen Hallada. Over the years a few things were tweaked on the test, chorus was added to the D-verb in PT8 further confusing things. A common complaint of the D-verb test is that nobody is using 150 D-verb’s in their session. I wanted to put forth a way for users to have some comparisons.
I am most concerned about testing the system in worst case scenario. So, a 64 HW Buffer is my normal buffer size to test as its where I track. This is simulating closer to a punch in scenario where the session has lots of plugins and I need to punch in a take on something. This is also when the system is most volatile. Problems will show pretty quick this way. DPC latency from bad drivers, running background processes or software, etc. will all greatly affect the performance. I know a system will perform best at 1024, so it is a bit less important IMO, except to see the “top end” of the system for the best-case scenario. Pro Tools is not quite as bad as it was years ago, about huge differences in performance between 64 and 1024, but it is still present. Pro Tools changing the buffer handling internally to only affect the input greatly changed how it worked previous to version 11.

Inadequacies in the driver of your audio interface is another topic. Not all drivers are built the same. Test results can fluctuate on the same system when using different drivers or interfaces. Then you have differences in operating systems such as Windows 7 to 10 or Yosemite to El Capitan that can affect the performance. Also, changes within Pro Tools itself can fluctuate the results. It is not going to be Apples to Apples when comparing say the most recent release 12.7.1 with version 11.3.2, or even 12.4 in that case.

On With The Testing

First, here is a breakdown of the D-verb test. The ultimate goal of this is to continually push the CPU of your system until the audio starts distorting or it errors out within a 5-minute time frame. The errors are typically either a -9073 or “AAE can’t get audio from the drives fast enough” errors. My issue with the D-verb test is having to listen to a 1k sine wave for the distorting and breaking up!

D-Verb Test

  1. Turn your interface audio DOWN
  2. Create a 24/48k session
  3. Set your playback buffer to 64
  4. Make sure “Ignore errors” is not checked
  5. Open the system usage window (which is under “Windows”)
  6. Create a mono audio track. 
  7. Set the output to buss 1. Set the input to “none”
  8. Print a 5 minute, 1k sine wave on a new mono audio track (Do this by selecting 5+ minutes inside the mono audio track).
  9. Go to the Audiosuite menu and select the signal generator and hit “process”
  10. Create a mono audio track
  11. Set the input to buss 1, output to your interface output.
  12. Instantiate 5 mono D-Verbs on that track.
  13. Duplicate this track roughly 50 times to begin with.
  14. Record arm the tracks and “OK” through the warnings. (make sure the main source track is not record armed)
  15. Turn up your audio interface a tiny bit to allow you to hear the playback and listen for pops and clicks.

From here the goal is to record the sine wave on as many tracks as possible for 10+ minutes while monitoring the stereo track. If you hear any crackling or distorting in the audio or receive any warning popup windows such as -9073 errors, Stop the session and remove a few tracks and try again. 

The goal here is to achieve perfect playback without errors or distortion for a 10 minute period.

An Alternative Test

The alternate test I came up with affects the system a bit differently and is a little bit more representative of a real world session (as well as being a little more pleasing on the ears). It uses a mix of Avid and Air plugins and is a bit more complex to setup so I am including a Pro Tools session file to download saving you the trouble.

Performance 2 Test

  1. Turn your interface audio DOWN
  2. Create a 24/48k session
  3. Set your playback buffer to 64
  4. Make sure “Ignore errors” is not checked
  5. Open the system usage window which is under “Windows”
  6. Create a Stereo audio track, 
    a) Set the input to None, output to your main interface outputs,
    b) Setup a send to Buss 1-2,
    c) Hit the “Pre” button on the send to make it pre-fader,
    d) Set the volume of the send to 0,
    e) Drag in 10+ minutes of various audio (Clean audio is preferred. Stay away from tracks with distorted instruments).
  7. Create another stereo audio track.
    a)Set the input to Buss 1-2, output to the main interface outputs.
    b)Turn the volume on the track all the way down (-inf)
    c)Instantiate a long set of plugins in inserts A-J
    A-E. Dither/BF-76/D-Verb/Channel Strip/AIR Kill EQ
    F-J. ModDelay 3/EQ3 7band/PSA-1(Sansamp)/Maxim/MasterMeter

    d)In the edit Window on the track, select the volume view
    e)Select the pencil and set it to random. Draw random volume automation for 10+ minutes.

  8. Duplicate this track about 20 times to start.
  9. Record arm these tracks and “OK” through the warnings. (make sure the main source track is not armed)

You should be monitoring the original source track while the tracks being recorded are turned down and not being monitored. Once the system hits its breaking point, the source track being monitored will crack and distort or you will start receiving system errors such as 9073.

In Closing

So, with the above test, it gives us a fairly reliable way to compare different systems performance. These tests are actually fairly quick to put together, but I am including links to session files to help speed up the process. A video is also linked here to help you through the steps if you wish to set it up yourself. If you have questions about how your system stacks up to a new machine you may be looking at, run the tests, and let us know the results and we can give you an idea.

Sign up to our email list, Follow us on Social Media and leave us your thoughts and comments below.

Follow Us !

Posted in Discussion, Technical Article, Tutorials and tagged , , , , .