EE109 – Spring 2023 Introduction to Embedded Systems

EE109 – Spring 2023: Introduction to Embedded Systems

Navigation menu

MacOS 64-bit Toolchain

Versions of macOS from 10.15 (a.k.a. "Catalina") and later have made it necessary to use an AVR development toolchain that consists of 64-bit executable programs. The previous toolchain used in EE109 consisted of 32-bit programs and can no longer be used. This web page describes a way to install a working AVR 64-bit toolchain based on using the Arduino IDE software. The Arduino IDE application actually contains a nearly complete copy of most of the AVR toolchain programs and with a little work these can be run from a Terminal window. Since Arduino appears to be periodically releasing new versions of their IDE, and this includes updated versions of the AVR toolchain, you can take advantage of this and use the toolchain software from within the Arduino IDE from a Terminal window.

Important: The information below assumes you are at least moderately familiar with using the Terminal program and its command line interface, and with basic Unix-type commands for moving around the file system and doing simple operations. If you are not familiar with using the command line interface we suggest you take a look at this web page for a short tutorial on the subject.

Warning Number 1: Some of the steps below require administrator privileges. When using the Terminal program this is done with the "sudo" command to run single command as the administrator. The first time the "sudo" is typed, the system will ask for your password, meaning the password you use to login to the Mac. When typing the password on the keyboard, the characters are not shown on the screen, nor will it show little black circles indicating characters have been typed. It will look like nothing has been typed, but finish entering the password and then press Return. Subsequent uses of "sudo" may not ask for the password if done shortly after the first one was done.

Warning Number 2: Starting in version 11 of the macOS ("Big Sur") Apple has added extra security "features" that can interfere with using the Terminal application for this class. To see if you are affected by this, open a Terminal window and type the command "ls Desktop". If you see a list of the files that are visible on the desktop screen, you can skip the rest of this section. However if you get a warning about the operation not permitted, you will need to follow the instructions below to fix things.

  1. Open the System Preferences window, and then click on the "Security and Privacy" icon.
  2. In the Security and Privacy window, click on the "Privacy" tab.
  3. Click on the small padlock in the lower left and then enter your password for the Mac to allow you to make changes.
  4. Scroll through the list in the left box to find "Full Disk Access" and click on it.
  5. Click on the plus sign below the box on the right. This will bring up a file selection dialog.
  6. Use the file selection dialog to go the top level Applications folder, and in there into the Utilities folder.
  7. In the Utilities folder click on Terminal.app and then click on Open at the bottom right of that screen. This should add the Terminal program to the list of appications in the box on the right.
  8. If there isn't one there, click in the box to the left of Terminal to put a check mark.
  9. Click on the open padlock to close it and then close the Security and Privacy window.

Once the change above has been made, try doing the "ls Desktop" test again to see if the desktop files can be seen.

Warning Number 3: The Apple Xcode developer tools are not required in order to run the avr-gcc software. If during the installation steps below the Mac asks about installing Xcode, you can cancel that since it's not needed. If Xcode has already been installed it should not affect running the avr-gcc software.



Step 0: Remove Any Old Toolchain Package

If you had previously installed the AVR toolchain from the "CrossPack" distribution, you should remove that toolchain so it doesn't get in the way of the programs you are installing below. If you aren't sure if that software is installed, check for it with the command line interface operation

    $ ls /usr/local

If you see any file or directories listed that start with "CrossPack-AVR" that means the CrossPack software has been installed and needs to be removed.

The CrossPack software has to be removed by using a command line interface operation. To remove it, use the command

    $ cd /usr/local
    $ sudo rm -rf  CrossPack-AVR-20131216  CrossPack-AVR

After that, try doing the "ls /usr/local/" operation again and check that the CrossPack entries have been removed.

Step 1: Install the Arduino IDE

Download and install version 1.8.19 of the Arduino IDE. On the Arduino software web page, you will have to scroll down to find this listed as "Legacy IDE" but this is the one you want to get. Do not download any version that starts with "2" or says it's a "release candidate" or "hourly build" since these are configured differently and will not work with our setup. Installing the Arduino app is just a matter of downloading the ".zip" file, unzipping it to get the Arduino software with the green, round icon, and then putting it someplace like the system Applications folder or in the Applications folder inside your home folder. It doesn't make any difference where it goes as long as you can find it using the Terminal program as described in the next step.

The Mac may have to be convinced to let you run the Arduino software. Double-click on the Arduino IDE application to start it up. It should first display the Arduino splashscreen and then a window where you can start typing a program. If that happens you can quit the program and move on the step 2 below. However if it instead puts up a warning that the Mac doesn't want to start the program since it's from an unknown developer, you will need to convince the Mac to run the program. From the Apple menu, open up the System Preferences and then click on the "Security & Privacy" icon, and on the screen that appears, click on the "General" tab. Towards the bottom of the window there should be place where you can confirm that you want to start the Arduino IDE application. Once the Arduino IDE starts, you can quit it and the issue should be fixed.

Step 2: Find the AVR Toolchain

Using a Terminal window, you'll need to find the AVR toolchain that is inside the Arduino IDE. First use the "cd" command to find the Arduino.app file and then do another "cd" to get inside the Arduino.app package. What looks like a single file on the Mac, like the Arduino.app package, is actually a directory full of subdirectories containing various things and the Terminal program can be used to get inside this directory structure. Assuming you installed it in the top level Applications folder do something like this.

    $ cd /Applications/Arduino.app

If the location is in a folder that has spaces in the path name, make sure to put quotes around the path. For example

    $ cd "/Users/ttrojan/My Apps/Arduino.app"

Now dig deeper into the directory structure to find the AVR toolchain.

    $ cd Contents/Java/hardware/tools/avr

This was the path to it as of the time this web page was written but it may change so be prepared to hunt for it if the above paths are no longer correct.

Doing an "ls" at this point should show that you are in the AVR toolchain directory and "ls bin" will show the subdirectory with the compiler, linker, downloader, etc.

    $ ls
    avr                             etc                             libexec
    bin                             include                         x86_64-apple-darwin16.1.0
    builtin_tools_versions.txt      lib

    $ ls bin
    arduinoOTA      avr-c++filt     avr-gcc-5.4.0   avr-gcov-tool   avr-man         avr-readelf
    avr-addr2line   avr-cpp         avr-gcc-ar      avr-gdb         avr-nm          avr-size
    avr-ar          avr-elfedit     avr-gcc-nm      avr-gprof       avr-objcopy     avr-strings
    avr-as          avr-g++         avr-gcc-ranlib  avr-ld          avr-objdump     avr-strip
    avr-c++         avr-gcc         avr-gcov        avr-ld.bfd      avr-ranlib      avrdude

Now that you have found the AVR toolchain files, you need to get the full path to the AVR toolchain. This can be done by typing the command "pwd -P" which will then print out your current location in the file system. Make a copy of this output string (just the output that was printed by the command, not the "pwd -P" part) for pasting later.

    $ pwd -P
    /Applications/Arduino.app/Contents/Java/hardware/tools/avr

Step 3: Install Link to AVR Toolchain

Put a link to the above AVR toolchain directory someplace where you can add it to the executable search path. A recommended place is in the "/usr/local" directory. Move to the /usr/local directory and make a symbolic link using the path to the AVR toolchain that you copied above. Make sure to put the path in quotes if it contain any spaces in the directory names. In the example below we also create a "/usr/local/bin" directory since we'll need that later.

         cd /usr/local
         sudo ln -s /Applications/Arduino.app/Contents/Java/hardware/tools/avr avr
         sudo mkdir bin
         cd $HOME

Note that on this command line after you paste in the path to the files in the Ardunino application, there is a space and "avr" at the very end of the command that you will have to add manually.

Important: The link you have just made is for wherever the Arduino application is currently located on your system. If you move the program into a different folder you will have to redo the above steps so the "avr" link is pointing to the new location.

You now have a "/usr/local/avr" directory with the toolchain programs. The last command above ("cd $HOME") does a "cd" back to your home directory.

Step 4: Adjust PATH Variable

The program that runs inside the Terminal window to accept typed commands and then do something is called the "shell" program. Whenever a command is typed, the shell has to find that program for the system to execute. The environment variable "PATH" contains a list of directories that the shell will search through when you tell it to execute a program. We can see the current setting of the PATH variable with the command "printenv PATH" which will print out the list of directories, separated by colons (":"), that will be searched. The list typically looks something like this, but may be different if you have installed other software packages.

    $ printenv PATH
    /usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin

In this case the system will search the "/usr/local/bin" directory for the program, then the "/usr/bin" directory, etc., down the list until if find the program in one of the directories and runs it. The default value of the PATH variable is set by the system, but often modified by user-written commands that get run when the shell starts up.

To make the Arduino avr-gcc toolchain work properly it's very important the PATH variable be set correctly so it includes the directories where the avr-gcc programs have been installed. The easiest way to do this is to edit one of the files in your home directory that the shell program refers to when starting to include a command that adds the correct directories to the list. Unfortunately there are several shell programs available for use with macOS, with names like "bash", "zsh", "tcsh", etc., and the choice of which file to edit depends on the shell your Mac is using. You need to determine which shell program you are using in order to proceed with the the installation. This can be done by displaying the value of the "SHELL" environment variable by typing the command "printenv SHELL". For example,

    $ printenv SHELL
    /bin/bash

shows that the "bash" shell is being used. Once you identify which shell program you are using you can follow the steps below.

To open one of these files with the Visual Studio Code editor, use the "Open..." item under the "File" menu and navigate to your home directory. Files that start with a period, like the one you are trying to edit, are hidden and don't normally appear in the list of files to open. To find them, hold down the command (⌘) and shift keys, and type a period ('.'). The list of files shows should refresh and now show all the hidden files that start with a period. You can select the one you need to open.

To open one of these files with the BBedit editor, use the "Open..." item under the "File" menu and navigate to your home directory. Files that start with a period, like the one you are trying to edit, are hidden and don't normally appear in the list of files to open. To find them, click on the "Options" button at the bottom of the box showing the files, and then click the "Show hidden items". The hidden files should now appear and you can select the one you need to open.

Edit the file and add this line to it.

    export PATH=/usr/local/bin:/usr/local/avr/bin:$PATH

This adds the /usr/local/bin and /usr/local/avr/bin directories to the start of the executable search path. Make sure to save the modified file.

IMPORTANT!: Once you have made the change to the files that sets the PATH variable, close the Terminal window you were using and open up a fresh one. Changes to the PATH variable only take effect when a new Terminal window is opened. To confirm that the path is now correct you can print it out.

    $ printenv PATH
    /usr/local/bin:/usr/local/avr/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin

It should show /usr/local/bin as the first directory, and /usr/local/avr/bin after it. If the /usr/local/bin directory shows up twice like in the example above you can ignore this since it won't affect things.

Step 5: Install the "avrdude" Program

The "avrdude" program for downloading code to the AVR microcontroller that comes with the Arduino IDE works when used directly from a Terminal window but it needs to be told where the "avrdude.conf" configuration file is located. The simplest fix for this is to create an "avrdude" shell script that invokes the Arduino IDE's "avrdude" program and supplies the path to the configuration file as an argument on the command line.

Download the file "macos-avrdude.zip" from the class web site (Click here). If it asks you where to put the file, find the "Desktop" folder put the file there. If it doesn't ask, it will probably put the .zip file in your "Downloads" folder. In that case use the graphical user interface to move the .zip file from the Downloads folder to the Desktop. Once the .zip file is on the Desktop, find it and double-click on it to extract the "avrdude.sh" file (unless the Mac has already done that for you).

You can put this shell script anywhere but it needs to be in a directory in your search path that is ahead of the /usr/local/avr/bin directory where the real "avrdude" program is located. In step 3 above we created a "/usr/local/bin" directory, and in step 4 we put that directory ahead of the /usr/local/avr/bin directory in the PATH variable, so that would be the preferred place to put the script.

In the Terminal windows, do the following steps to install the file in /usr/local/bin and make it executable.

    $ cd $HOME/Desktop
    $ chmod +x avrdude.sh
    $ sudo mv avrdude.sh /usr/local/bin/avrdude
    $ xattr -d com.apple.quarantine /usr/local/bin/avrdude
    $ cd $HOME

Note: If the "xattr" command above doesn't work, just ignore this since Apple may have changed their OS that used to require this step.

Once the above steps have been done to create the "avrdude" script, when you run "avrdude", it will cause the three line "avrdude" file you created to execute the real "avrdude" program in the Arduino IDE and supply the correct arguments to it. Since it's not in the Arduino.app folder, it's not affected by updating to a newer Arduino.app and should work with any newer version.

If you try typing the command avrdude at this point, it may put display some lines about options available when invoking the program, or it may display an error messages about no programming device specified. In either case this shows the program is being executed and you can move to the next step.

Important: The above will only work if your PATH variable is such that the copy of avrdude you just created is found before the one in /usr/local/avr/bin. If you followed the instructions in step 4 above this should be the case. Otherwise you may have to make adjustments to the PATH, or put avrdude someplace else to make it all work.

Step 6: Install a Working "make" Program

In order to compile and download the programs to your Arduino, a working version of the "make" utility needs to be on your system. Apple usually provides this as part of the system software, but in some releases of the macOS, the "make" program that comes with it is mostly non-functional. In the Terminal window, type the command "make --version" and see what happens.

If it responds with a some lines showing the version number and some copyright notices, like this

    $ make --version
    GNU Make 3.81
    Copyright (C) 2006  Free Software Foundation, Inc.
    This is free software; see the source for copying conditions.
    There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

    This program built for i386-apple-darwin11.3.0

then the "make" provided by Apple will probably work fine and you can skip the rest of this step and proceed to Step 7.

If it responds with a message about how Apple's developer tools need to be installed like this

    $ make --version
    xcode-select: note: no developer tools were found at '/Application/Xcode.app',
    requesting install. Choose an option in the dialog to download the command line
    developer tools.

and may put up a dialog about "The 'make' command requires the command line developer tools.", then click the "Not now" button in the dialog and follow the steps below to install a working version of "make".

There are a few different ways to get a working "make" on the system.

If you decide to use the pre-complied "make" from our web site, click on the link above to download the "macos-make.zip" file in the same manner you downloaded the "macos-avrdude.zip" file in the previous step. If it asks, put the .zip file in "Desktop". If it doesn't ask, it will likely end up in the "Downloads" folder and you can use the graphical use interface to move the .zip file from there the Desktop. Once it's on the Desktop, double-click on it to extract the file "make" (unless the Mac has already done that for you). The executable "make" program now needs to be installed in some place like /usr/local/bin.

    $ cd $HOME/Desktop
    $ chmod +x make
    $ sudo mv make /usr/local/bin/
    $ cd $HOME

Whatever directory "make" is installed in, that directory must be in the executable search path ahead of /usr/bin where Apple's "make" is normally installed.

If you decide to compile it yourself from Gnu sources, be advised that the normal way to build the "make" program from sources uses a working copy of "make" already on the system. Since you probably don't have that follow the instructions in the "README" file to use the provided "build.sh" script to build the "make" program. After the program is compiled, decide where to put the executable "make", perhaps /usr/local/bin, and move it there. Make sure whatever directory you put it in is also included in your executable search path. If not, edit the shell startup file as shown above to include it.

Step 7: Test Installation

At this point all the command line operations for compiling and downloading AVR software should be working. You might have to kill and re-open the Terminal window you used to install the software for some of it to work but after that it should work normally. First use the "which" command to confirm which program execute when you type the command. It will print out the program that will execute and check to make sure they are the ones in either /usr/local/bin or /usr/local/avr/bin. You can skip the test for the "make" program if you didn't install a new one in the section above.

    $ which make
    /usr/local/bin/make

    $ which avr-gcc
    /usr/local/avr/bin/avr-gcc

    $ which avrdude
    /usr/local/bin/avrdude

Next, you should be able to test running each program and have it print out their version numbers, which may not be exactly the same as those shown below. IMPORTANT: When you try running one of the programs, if a dialog appears on the screen about how the program "cannot be opened because the developer cannot be verified" follow the instructions at the end of this section to override this restriction, then come back and repeat the test to make sure the program can be run.

    $ make --version
    GNU Make 4.2.1
    Built for x86_64-apple-darwin15.5.0

    $ avr-gcc --version
    avr-gcc (GCC) 7.3.0

    $ avrdude --help
    (this prints lots of stuff but at the end...)
    avrdude version 6.3-20190619, URL: ....

Some of these commands may cause the Mac to put a box on the screen saying this program is from an unknown developer and it refuses to run it. If that happens, go to the "System Preferences" under the Apple menu and click on the "Security & Privacy" icon. In the bottom half of the screen is should list the program you tried to run and give you the option to allow running it. When you try running it again it may put up another box asking if you sure you want to run it. Confirm that you do and the program will probably run. After all that the program will run in the future without asking.

Done!

At this point the installation of the AVR toolchain is complete. Please return to the previous web page and continue with the installation of the rest of the development software.