SPO600 Project - Stage 3: Analysis

 Stage 3 Summary

This is my summary post for Stage 3 of the Winter 2022 SPO600 Project. This post will be an analysis of the changes I made to SoX, command line utility that can convert various formats of computer audio files in to other formats. My changes were to implement SVE2 optimizations using autovectorizations. To see the outline for this project click here. To see my previous blog posts on this project, see the links below.

Stage 1 Summary

Stage 2 Summary

Here are some more useful links:

This is my git repo where I store some of my tests. It is seperated in two folders, changes and noChanges. The former being files created with SoX with my changes active, and the latter being files I made with none of my changes active. Included in these are audio files I converted, and disassemblies of two files where SVE2 instructions were implemented. 

This is my git repo of SoX, which I'm actively making my changes in.


Disassembly Analysis

In stage 2 to check that my code worked, I searched for the whilelo instruction (an SVE2 instruction) within SoX's code. I found that two files had been affected, libsox.so.3.0.0 and sox, both in the .libs folder. There were 376 whilelo's in libsox.so.3.0.0 and 2 in sox. For this stage I've made a disassembly of both of these files, with and without my changes. I've uploaded these disassemblies into my git repo for test files. Disassemblies for sox are named soxDisassembly.txt and ones for libsox.so.3.0.0 are named libSoxDisassembly.txt. The immediate conclusion to draw, is that the libsox.so.3.0.0 disassembly is huge. 85515 lines without my changes and 98363 with my changes. Not really a conclusion to be drawn here I'm using this as my reasoning for why I'm going to focus on sox, which has a far more reasonable 7845/8395 lines for without changes/with changes. Now honestly, I don't know what to say. I don't really understand my changes. I can tell you that both whilelo's that appear in sox appear under update_status, which is at line 3926 in the disassembly with my changes, and line 4700 in the disassembly without my changes. But for understanding what's happening, I just feel out of my depth, and I don't like saying that. But I really have no idea what my changes really mean, and I have no idea how to explain what impact I think this will have on the code. 


Testing

Included with SoX is a audio file named monkey.wav. The file is just a screech of a monkey, but SoX offers it as a way to test the code. I've made a git repo of my tests with this file, as I converted it into other audio formats. I was only able to test the aif, aiff, and aifc file on my macbook, but all files still worked the same. To be sure that SoX was using the SVE2 code, I removed qemu-aarch64, which is code that I added to SoX's script to have the processor emulate SVE2 instructions, since the processor I am using can't run SVE2 instructions naturally. The result of trying to run these SoX tests without this code was a very positive Illegal instruction (core dumped). Great news because that's what happens when the processor tries to run SVE2 instructions without the emulation turned on.

The other big test I am able to do is also provided with SoX, and it's a file named testall.sh. Previously in my stage 2 I compared the output of testall.sh before my changes and after my changes to prove my code worked fine, as the output for both was exactly the same, But I did fail to try running testall.sh while SVE2 emulation was disabled, so that's what I did now. So right after doing the tests mentioned above, I ran testall.sh and got:

Format: 8svx   Options: 

./testall.sh: line 39: 527312 Illegal instruction     (core dumped) ${bindir}/sox${EXEEXT} ${srcdir}/monkey.wav $opts /tmp/monkey.$format $effect

./testall.sh: line 39: 527328 Illegal instruction     (core dumped) ${bindir}/sox${EXEEXT} $opts /tmp/monkey.$format /tmp/monkey1.wav $effect

Format: aiff   Options: 

./testall.sh: line 39: 527349 Illegal instruction     (core dumped) ${bindir}/sox${EXEEXT} ${srcdir}/monkey.wav $opts /tmp/monkey.$format $effect

./testall.sh: line 39: 527377 Illegal instruction     (core dumped) ${bindir}/sox${EXEEXT} $opts /tmp/monkey.$format /tmp/monkey1.wav $effect

Format: aifc   Options: 

./testall.sh: line 39: 527399 Illegal instruction     (core dumped) ${bindir}/sox${EXEEXT} ${srcdir}/monkey.wav $opts /tmp/monkey.$format $effect

./testall.sh: line 39: 527425 Illegal instruction     (core dumped) ${bindir}/sox${EXEEXT} $opts /tmp/monkey.$format /tmp/monkey1.wav $effect

Format: au   Options: 

./testall.sh: line 39: 527447 Illegal instruction     (core dumped) ${bindir}/sox${EXEEXT} ${srcdir}/monkey.wav $opts /tmp/monkey.$format $effect

./testall.sh: line 39: 527472 Illegal instruction     (core dumped) ${bindir}/sox${EXEEXT} $opts /tmp/monkey.$format /tmp/monkey1.wav $effect

Format: avr   Options: -e unsigned-integer

./testall.sh: line 39: 527496 Illegal instruction     (core dumped) ${bindir}/sox${EXEEXT} ${srcdir}/monkey.wav $opts /tmp/monkey.$format $effect

./testall.sh: line 39: 527523 Illegal instruction     (core dumped) ${bindir}/sox${EXEEXT} $opts /tmp/monkey.$format /tmp/monkey1.wav $effect

Format: cdr   Options: 

./testall.sh: line 39: 527548 Illegal instruction     (core dumped) ${bindir}/sox${EXEEXT} ${srcdir}/monkey.wav $opts /tmp/monkey.$format $effect

./testall.sh: line 39: 527575 Illegal instruction     (core dumped) ${bindir}/sox${EXEEXT} $opts /tmp/monkey.$format /tmp/monkey1.wav $effect

Format: cvs   Options: 

./testall.sh: line 39: 527596 Illegal instruction     (core dumped) ${bindir}/sox${EXEEXT} ${srcdir}/monkey.wav $opts /tmp/monkey.$format $effect

./testall.sh: line 39: 527622 Illegal instruction     (core dumped) ${bindir}/sox${EXEEXT} $opts /tmp/monkey.$format /tmp/monkey1.wav $effect

Format: dat   Options: 

./testall.sh: line 39: 527647 Illegal instruction     (core dumped) ${bindir}/sox${EXEEXT} ${srcdir}/monkey.wav $opts /tmp/monkey.$format $effect

./testall.sh: line 39: 527672 Illegal instruction     (core dumped) ${bindir}/sox${EXEEXT} $opts /tmp/monkey.$format /tmp/monkey1.wav $effect

Format: hcom   Options: -r 22050

./testall.sh: line 39: 527693 Illegal instruction     (core dumped) ${bindir}/sox${EXEEXT} ${srcdir}/monkey.wav $opts /tmp/monkey.$format $effect

./testall.sh: line 39: 527718 Illegal instruction     (core dumped) ${bindir}/sox${EXEEXT} $opts /tmp/monkey.$format /tmp/monkey1.wav $effect

Format: maud   Options: 

./testall.sh: line 39: 527743 Illegal instruction     (core dumped) ${bindir}/sox${EXEEXT} ${srcdir}/monkey.wav $opts /tmp/monkey.$format $effect

./testall.sh: line 39: 527768 Illegal instruction     (core dumped) ${bindir}/sox${EXEEXT} $opts /tmp/monkey.$format /tmp/monkey1.wav $effect

Format: prc   Options: 

./testall.sh: line 39: 527793 Illegal instruction     (core dumped) ${bindir}/sox${EXEEXT} ${srcdir}/monkey.wav $opts /tmp/monkey.$format $effect

./testall.sh: line 39: 527817 Illegal instruction     (core dumped) ${bindir}/sox${EXEEXT} $opts /tmp/monkey.$format /tmp/monkey1.wav $effect

Format: prc   Options: -e signed-integer

./testall.sh: line 39: 527838 Illegal instruction     (core dumped) ${bindir}/sox${EXEEXT} ${srcdir}/monkey.wav $opts /tmp/monkey.$format $effect

./testall.sh: line 39: 527863 Illegal instruction     (core dumped) ${bindir}/sox${EXEEXT} $opts /tmp/monkey.$format /tmp/monkey1.wav $effect

Format: sf   Options: 

./testall.sh: line 39: 527888 Illegal instruction     (core dumped) ${bindir}/sox${EXEEXT} ${srcdir}/monkey.wav $opts /tmp/monkey.$format $effect

./testall.sh: line 39: 527914 Illegal instruction     (core dumped) ${bindir}/sox${EXEEXT} $opts /tmp/monkey.$format /tmp/monkey1.wav $effect

Format: smp   Options: 

./testall.sh: line 39: 527939 Illegal instruction     (core dumped) ${bindir}/sox${EXEEXT} ${srcdir}/monkey.wav $opts /tmp/monkey.$format $effect

./testall.sh: line 39: 527964 Illegal instruction     (core dumped) ${bindir}/sox${EXEEXT} $opts /tmp/monkey.$format /tmp/monkey1.wav $effect

Format: sndt   Options: 

./testall.sh: line 39: 527985 Illegal instruction     (core dumped) ${bindir}/sox${EXEEXT} ${srcdir}/monkey.wav $opts /tmp/monkey.$format $effect

./testall.sh: line 39: 528010 Illegal instruction     (core dumped) ${bindir}/sox${EXEEXT} $opts /tmp/monkey.$format /tmp/monkey1.wav $effect

Format: txw   Options: 

./testall.sh: line 39: 528036 Illegal instruction     (core dumped) ${bindir}/sox${EXEEXT} ${srcdir}/monkey.wav $opts /tmp/monkey.$format $effect

./testall.sh: line 39: 528062 Illegal instruction     (core dumped) ${bindir}/sox${EXEEXT} $opts /tmp/monkey.$format /tmp/monkey1.wav $effect

Format: ub   Options: -r 8130

./testall.sh: line 39: 528083 Illegal instruction     (core dumped) ${bindir}/sox${EXEEXT} ${srcdir}/monkey.wav $opts /tmp/monkey.$format $effect

./testall.sh: line 39: 528108 Illegal instruction     (core dumped) ${bindir}/sox${EXEEXT} $opts /tmp/monkey.$format /tmp/monkey1.wav $effect

Format: vms   Options: 

./testall.sh: line 39: 528132 Illegal instruction     (core dumped) ${bindir}/sox${EXEEXT} ${srcdir}/monkey.wav $opts /tmp/monkey.$format $effect

./testall.sh: line 39: 528159 Illegal instruction     (core dumped) ${bindir}/sox${EXEEXT} $opts /tmp/monkey.$format /tmp/monkey1.wav $effect

Format: voc   Options: 

./testall.sh: line 39: 528183 Illegal instruction     (core dumped) ${bindir}/sox${EXEEXT} ${srcdir}/monkey.wav $opts /tmp/monkey.$format $effect

./testall.sh: line 39: 528207 Illegal instruction     (core dumped) ${bindir}/sox${EXEEXT} $opts /tmp/monkey.$format /tmp/monkey1.wav $effect

Format: vox   Options: -r 8130

./testall.sh: line 39: 528229 Illegal instruction     (core dumped) ${bindir}/sox${EXEEXT} ${srcdir}/monkey.wav $opts /tmp/monkey.$format $effect

./testall.sh: line 39: 528254 Illegal instruction     (core dumped) ${bindir}/sox${EXEEXT} $opts /tmp/monkey.$format /tmp/monkey1.wav $effect

Format: wav   Options: 

./testall.sh: line 39: 528279 Illegal instruction     (core dumped) ${bindir}/sox${EXEEXT} ${srcdir}/monkey.wav $opts /tmp/monkey.$format $effect

./testall.sh: line 39: 528306 Illegal instruction     (core dumped) ${bindir}/sox${EXEEXT} $opts /tmp/monkey.$format /tmp/monkey1.wav $effect

Format: wve   Options: 

./testall.sh: line 39: 528327 Illegal instruction     (core dumped) ${bindir}/sox${EXEEXT} ${srcdir}/monkey.wav $opts /tmp/monkey.$format $effect

./testall.sh: line 39: 528352 Illegal instruction     (core dumped) ${bindir}/sox${EXEEXT} $opts /tmp/monkey.$format /tmp/monkey1.wav $effect

A whole bunch of dumped cores. Inspiring really. For comparison I will include the "ideal" output, that being the output I got before making the changes, and also the output I got when I emulate SVE2 instructions.

Format: 8svx   Options: 

Format: aiff   Options: 

Format: aifc   Options: 

Format: au   Options: 

Format: avr   Options: -e unsigned-integer

Format: cdr   Options: 

/home/pzdanko/project/SoX/sox-code/src/.libs/sox WARN formats: cdda can't encode at 8012Hz; using 44100Hz

/home/pzdanko/project/SoX/sox-code/src/.libs/sox WARN formats: cdda can't encode mono; setting channels to 2

Format: cvs   Options: 

Format: dat   Options: 

Format: hcom   Options: -r 22050

Format: maud   Options: 

Format: prc   Options: 

/home/pzdanko/project/SoX/sox-code/src/.libs/sox WARN formats: prc can't encode at 8012Hz; using 8000Hz

Format: prc   Options: -e signed-integer

/home/pzdanko/project/SoX/sox-code/src/.libs/sox WARN formats: prc can't encode at 8012Hz; using 8000Hz

/home/pzdanko/project/SoX/sox-code/src/.libs/sox WARN formats: prc can't encode Signed Integer PCM

Format: sf   Options: 

Format: smp   Options: 

Format: sndt   Options: 

Format: txw   Options: 

/home/pzdanko/project/SoX/sox-code/src/.libs/sox WARN formats: txw can't encode at 8012Hz; using 16666.7Hz

Format: ub   Options: -r 8130

/home/pzdanko/project/SoX/sox-code/src/.libs/sox WARN raw: `/tmp/monkey.ub': # channels not specified; trying mono

Format: vms   Options: 

Format: voc   Options: 

Format: vox   Options: -r 8130

Format: wav   Options: 

Format: wve   Options: 

So I can draw from this that my tests do for sure involve the SVE2 instructions, aka the changes I made through autovectorization, and SoX's functionality is not broken by those changes. At least on the aarch64 platform. Which reminds me-


Other Platforms

So this is the part where I would like to talk about how my changes affected other platforms, specifically the x86_64 platform. Of course the key words in that sentence is "I would like to". Unfortunately I didn't have any luck. I know I talked about this in stage 2, but I wanted to bring it up again because I did want to emphasize that I have tried getting SoX to run on x86_64. As I mentioned before, I can't even get SoX to run before making any changes. I wasn't able to make time this week to talk to the professor this week due to exams and other projects, but hopefully you can take my word that I did try my best to test this, and I wasn't sure what other platforms I could try to test it on besides the two provided to us.


Summary

In summary, I had a rough time with this stage. I'm fairly confident that my changes have taken affect, and that they haven't compromised SoX's functionality, at least on aarch64. But trying to understand the huge amount of code and what my changes meant was really difficult, and especially this being during exam week it was pretty impossible for me to make any time to discuss this stage with the professor. If I had to make some recommendation for how to continue my work it would be to gain some better understanding of the code. That was a semi-sad joke at my expense. Anyways here's to finishing SPO600 I guess. Cheers.






Comments

Post a Comment

Popular posts from this blog

SPO600 Project - Stage 1: Selection

Hello again, this post is my summary for Stage 1 of my SPO600 project. I'll be explaining the project in this post but the instructions for this project can be found here . The premise of this project is to add Scalable Vector Extension version two (SVE2) optimizations to an existing open source project. An introduction to SVE2 can be found here . To keep it incredibly short, SVE2 is a new, minor refinement over SVE, which itself is an improvement of SIMD (Single Instruction Multiple Data) optimizations. These are optimizations that improve code that process multitudes of data through a single instruction. Sorry I know this probably isn't the best explanation but that's the best I can do to give new readers some context. So with that let's dive into my project. Step 1 - Candidate Identification The first step is to identify some candidates to optimize. As a starting point to look for these candidates, I based my search around video games and the libraries these games us...
Read more

SPO600 - Final Thoughts

Hey here's a last minute blog post to try and bump up the number of blog posts for April. It's really just a sort of a farewell to this course and some of my thoughts on it.  Overall this course was really neat. I had never worked with assembly language code before and it was certainly something. Honestly it feels really restrictive and generally difficult to work with but it was definitely an enjoyable experience. Though I can't say it's something I want to continue in. But learning more about the compiler and it's optimizations is definitely something I'll take away from this course. I never had the compiler explained to me in any depth before, hell I didn't even realize -Wall meant turning on all warnings. I just always read it very literally as wall and gave it no thought. It was always just something I was told to type to run my program but no one had ever offered any deeper explanation into it (nor did I even think to ask) so it was a cool thing to lea...
Read more

Lab 4 - 64-bit Assembly Language Lab - Part 4

 Welcome to part 3 of lab 4. This lab goes over some basics of 64-bit assembly languages, specifically in the AArch64 and x86_64 platforms. This specific part will be going over the second half of the lab in the AArch64 system. Links for the other parts are listed here: Part 1 - First half in AArch64 Part 2 - First half in x86_64 Part 3 - Second half in AArch64 I'll be outlining the steps in detail as I go along but the full instructions for this lab can be found here . Okay we're in the homestretch. Just like part 3 we're making our loop print from 00-30. I learned my lesson from part 3 and pre-labelled my code, and I was already planning on trying to reduce the overall amount of registers I use but x86_64 seems to have less safe registers anyways so I was somewhat forced to. Major difference here is the div function. The div function in x86_64 calculates the remainder too so that was pretty convenient. It's a bit less intuitive, it takes one argument, a register, a...
Read more