SPO600 Project - Stage 2: Implementation

 Stage 2 Summary

This is my summary post for Stage 2 of the Winter 2022 SPO600 Project. To see my Stage 1 post click here. To see the outline for this project click here.

A quick summary of my Stage 1 was I planned to implement SVE2 optimizations in SoX by using autovectorization. In this stage I will be attempting to implement those changes. 

Here are some more useful links:

This is my repo where I store the audio files I've made to test SoX. The noChanges folder is for audio files converted before my changes, and the changes folder is for audio files converted after my changes.

This is my repo of SoX, which I'm actively making my changes in.

Step 1: Implementing Changes

First step: implementing those changes. Since I've chosen to do autovectorization this part is honestly a bit quick. I change the CFLAGS variable in SoX's configure script from optimization level 2 to level 3, and add an option to include SVE2 extensions. The original CLFAGS value and the updated one are offered below.

original:

-g -O2 -fstack-protector-strong -Wall -Wmissing-prototypes -Wstrict-prototypes -fopenmp

edited: 

-g -O3 -march=armv8-a+sve2 -fstack-protector-strong -Wall -Wmissing-prototypes -Wstrict-prototypes -fopenmp

For these changes I've decided to use O3 for the optimizations, but if I run into any issues I plan to use O2 with ftree-vectorize as a back-up. But so long as O3 does not cause any apparent issues during testing I figured why not. The more optimizations the better I think.

Anyways after this I run the configure script and I have my makefiles all created with the proper control options, and I run make. Once this is done I try to run SoX and thankfully it crashes. Which actually is good news because this likely means my changes were implemented and the autovectorizer did find places to make optimizations, and since I wasn't running the code on a processor that supports SVE2. So for my final change I edit the shell script of SoX and add qemu-aarch64 to the section of the code that actually runs SoX. The section I edited is shown below. This addition is done so the processor can emulate SVE2 functionality. 

func_exec_program_core ()

{


      if test -n "$lt_option_debug"; then

        $ECHO "sox:sox:$LINENO: newargv[0]: $progdir/$program" 1>&2

        func_lt_dump_args ${1+"$@"} 1>&2

      fi

      exec qemu-aarch64 "$progdir/$program" ${1+"$@"}


      $ECHO "$0: cannot exec $program $*" 1>&2

      exit 1

}

With this I then try running SoX again and good news it works. At least on the surface level. Next step is to verify those changes a bit more thoroughly.


Step 2: Verifying Changes 

Now I'd like to preface this section by mentioning that with the new stage 3 requirements, I am a bit confused what to do here, as some of the analysis in Stage 3 does seem to overlap with how I imagined I would verify my changes built correctly. What I've decided is that I'll do some preliminary tests here to make sure SoX is still working as expected and in Stage 3 I'll take a deeper dive into what's happening.

So first of all, the first test already went great. SoX blew up when I wasn't emulating SVE2 and it ran happily when I was. Great news. Next is to try get SoX to actually do something useful. SoX comes with a sample audio file named Monkey.wav, which is a short monkey screech. Before making my changes I converted this file into a couple over audio files, namely: aiff, aif, and aifc. I point out these specifically as these are the files I'm able to actually play on my macbook, along with the original .wav file. With my changes now implemented I decide to convert the wav file again into these formats. I've listened to them all again and there doesn't seem to be any difference between the files from before the changes and after. If you'd like to see for yourself, I've made this git repo, with a folder for files converted with no changes and with changes.

The other test I'm able to do is provided with SoX, and that is a test script called testall.sh. I ran this script before and after making my changes, and after comparing the output from both they're both the same. That output is included below. Consider both outputs were exactly identical I've decided to consider it a success at this stage.

Format: 8svx   Options: 

Format: aiff   Options: 

Format: aifc   Options: 

Format: au   Options: 

Format: avr   Options: -e unsigned-integer

Format: cdr   Options: 

/home/pzdanko/project/SoX/sox-code/src/.libs/sox WARN formats: cdda can't encode at 8012Hz; using 44100Hz

/home/pzdanko/project/SoX/sox-code/src/.libs/sox WARN formats: cdda can't encode mono; setting channels to 2

Format: cvs   Options: 

Format: dat   Options: 

Format: hcom   Options: -r 22050

Format: maud   Options: 

Format: prc   Options: 

/home/pzdanko/project/SoX/sox-code/src/.libs/sox WARN formats: prc can't encode at 8012Hz; using 8000Hz

Format: prc   Options: -e signed-integer

/home/pzdanko/project/SoX/sox-code/src/.libs/sox WARN formats: prc can't encode at 8012Hz; using 8000Hz

/home/pzdanko/project/SoX/sox-code/src/.libs/sox WARN formats: prc can't encode Signed Integer PCM

Format: sf   Options: 

Format: smp   Options: 

Format: sndt   Options: 

Format: txw   Options: 

/home/pzdanko/project/SoX/sox-code/src/.libs/sox WARN formats: txw can't encode at 8012Hz; using 16666.7Hz

Format: ub   Options: -r 8130

/home/pzdanko/project/SoX/sox-code/src/.libs/sox WARN raw: `/tmp/monkey.ub': # channels not specified; trying mono

Format: vms   Options: 

Format: voc   Options: 

Format: vox   Options: -r 8130

Format: wav   Options: 

Format: wve   Options: 

/home/pzdanko/project/SoX/sox-code/src/.libs/sox WARN formats: wve can't encode at 8012Hz; using 8000Hz



The last "test" I'll do here is just to give some extra confirmation that SVE2 optimizations were actually made. To do this I look through the executable files of SoX looking for any instance of whilelo, an SVE2 instruction. some quick checks have me find that I have 376 whilelos in the /.libs/libsox.so.3.0.0 file and 2 in /.libs/sox file. In Stage 3 I'll dive deeper into this but for now all this gives me some good confirmation that my changes have been implemented and SoX is still running as expected.


Step 3: Testing Other Platforms

The last step of this stage is to confirm that the project still runs properly on other platforms. For this I decide to try my changes on the x86_64 platform. Unfortunately, I wasn't able to test my changes on this platform. Before even making my changes to SoX, I tried to get the regular SoX working on x86_64 and ran into some issues. Namely that before running the configure script I'm supposed to run the command autoreconf -i. This gives me an error shown below.

lpc10/Makefile.am:8: error: Libtool library used but 'LIBTOOL' is undefined

lpc10/Makefile.am:8:   The usual way to define 'LIBTOOL' is to add 'LT_INIT'

lpc10/Makefile.am:8:   to 'configure.ac' and run 'aclocal' and 'autoconf' again.

lpc10/Makefile.am:8:   If 'LT_INIT' is in 'configure.ac', make sure

lpc10/Makefile.am:8:   its definition is in aclocal's search path.

src/Makefile.am:26: error: Libtool library used but 'LIBTOOL' is undefined

src/Makefile.am:26:   The usual way to define 'LIBTOOL' is to add 'LT_INIT'

src/Makefile.am:26:   to 'configure.ac' and run 'aclocal' and 'autoconf' again.

src/Makefile.am:26:   If 'LT_INIT' is in 'configure.ac', make sure

src/Makefile.am:26:   its definition is in aclocal's search path.

autoreconf: automake failed with exit status: 1

I tried editing the configure.ac file to define LIBTOOL but honestly I couldn't get it to work. I wish I could offer more here but I really don't know what else to do for this step.



Comments

Post a Comment

Popular posts from this blog

SPO600 - Final Thoughts

Hey here's a last minute blog post to try and bump up the number of blog posts for April. It's really just a sort of a farewell to this course and some of my thoughts on it.  Overall this course was really neat. I had never worked with assembly language code before and it was certainly something. Honestly it feels really restrictive and generally difficult to work with but it was definitely an enjoyable experience. Though I can't say it's something I want to continue in. But learning more about the compiler and it's optimizations is definitely something I'll take away from this course. I never had the compiler explained to me in any depth before, hell I didn't even realize -Wall meant turning on all warnings. I just always read it very literally as wall and gave it no thought. It was always just something I was told to type to run my program but no one had ever offered any deeper explanation into it (nor did I even think to ask) so it was a cool thing to lea
Read more

SPO600 Project - Stage 1: Selection

Hello again, this post is my summary for Stage 1 of my SPO600 project. I'll be explaining the project in this post but the instructions for this project can be found here . The premise of this project is to add Scalable Vector Extension version two (SVE2) optimizations to an existing open source project. An introduction to SVE2 can be found here . To keep it incredibly short, SVE2 is a new, minor refinement over SVE, which itself is an improvement of SIMD (Single Instruction Multiple Data) optimizations. These are optimizations that improve code that process multitudes of data through a single instruction. Sorry I know this probably isn't the best explanation but that's the best I can do to give new readers some context. So with that let's dive into my project. Step 1 - Candidate Identification The first step is to identify some candidates to optimize. As a starting point to look for these candidates, I based my search around video games and the libraries these games us
Read more

Lab 4 - 64-bit Assembly Language Lab - Part 4

 Welcome to part 3 of lab 4. This lab goes over some basics of 64-bit assembly languages, specifically in the AArch64 and x86_64 platforms. This specific part will be going over the second half of the lab in the AArch64 system. Links for the other parts are listed here: Part 1 - First half in AArch64 Part 2 - First half in x86_64 Part 3 - Second half in AArch64 I'll be outlining the steps in detail as I go along but the full instructions for this lab can be found here . Okay we're in the homestretch. Just like part 3 we're making our loop print from 00-30. I learned my lesson from part 3 and pre-labelled my code, and I was already planning on trying to reduce the overall amount of registers I use but x86_64 seems to have less safe registers anyways so I was somewhat forced to. Major difference here is the div function. The div function in x86_64 calculates the remainder too so that was pretty convenient. It's a bit less intuitive, it takes one argument, a register, a
Read more