Re: [hatari-devel] Hatari macOS builds - handover need?

[ Thread Index | Date Index | More lists.tuxfamily.org/hatari-devel Archives ]


part 2

The post buildscript. This packages the app with all the needed files to be distributable.

#!/bin/bash
EXECFILE=${BUILT_PRODUCTS_DIR}/${EXECUTABLE_PATH}
LIBPATH=${BUILT_PRODUCTS_DIR}/${FRAMEWORKS_FOLDER_PATH}
NEWLIBPATH="@executable_path/../Frameworks"
FWPATH=${BUILT_PRODUCTS_DIR}/${FRAMEWORKS_FOLDER_PATH}
NEWFWPATH="@executable_path/../Frameworks"

# we link to fat libs in /opt/local/lib when building
# EXPLANATION: I use macports to fetch x86+aarch64 libraries for inclusion into the archive
install_name_tool -change /opt/local/lib/libz.1.dylib /usr/lib/libz.1.dylib ${EXECFILE}

mkdir -p ${LIBPATH}
cp ${SRCROOT}/libncurses.6.dylib ${LIBPATH}/
# space separated list of libraries
# EXPLANATION: This are the (dual architecture) libraries I include. They're prebuilt and kept in the build env.
TARGETS="libpng16.16.dylib libportmidi.dylib libreadline.8.1.dylib"
for TARGET in ${TARGETS} ; do
  cp ${SRCROOT}/${TARGET} ${LIBPATH}/
  LIBFILE=${LIBPATH}/${TARGET}
  TARGETID=`/Library/Developer/CommandLineTools/usr/bin/otool-classic -DX ${LIBPATH}/${TARGET}`
  NEWTARGETID=${NEWLIBPATH}/${TARGET}
  install_name_tool -id ${NEWTARGETID} ${LIBFILE}
  install_name_tool -change ${TARGETID} ${NEWTARGETID} ${EXECFILE}
done

mkdir -p ${FWPATH}
cp -a ${SRCROOT}/SDL2.framework ${FWPATH}/
cp -a ${SRCROOT}/CAPSImage.framework ${FWPATH}/
# space separated list of frameworks
# EXPLANATION: ... and the same for frameworks. Dual architecture, kept in the build env.
TARGETS="CAPSImage.framework/CAPSImage SDL2.framework/SDL2"
for TARGET in ${TARGETS} ; do
  FWFILE=${FWPATH}/${TARGET}
  TARGETID=`/Library/Developer/CommandLineTools/usr/bin/otool-classic -DX ${FWPATH}/${TARGET}`
  NEWTARGETID=${NEWFWPATH}/${TARGET}
  install_name_tool -id ${NEWTARGETID} ${FWFILE}
  install_name_tool -change ${TARGETID} ${NEWTARGETID} ${EXECFILE}
done

# TOS also
mkdir -p ${BUILT_PRODUCTS_DIR}/${CONTENTS_FOLDER_PATH}/Resources
cp ${SRCROOT}/tos.img ${BUILT_PRODUCTS_DIR}/${CONTENTS_FOLDER_PATH}/Resources

# and strip
strip ${EXECFILE}




------- Original Message -------
On Thursday, April 27th, 2023 at 10:46 AM, Troed Sångberg <troed@xxxxxxxxxxx> wrote:

Warning, long & formatted emails, part 1

Code and comments ("EXPLANATION:") on how my nightly builds work (it's close to the release builds). With the discussion you're already having maybe it's enough to gain some insight from this instead of blindly replicating my code. It's been in use for a long time now and can surely be made better. Helper files in subsequent emails. I don't intend for anyone to copy this out, I will supply the VM where all of this is included already - this is to add to the discussion you're already having.

This script is run every X minutes from cron. Triggering a similar script on commit as a CI build would be way better.

#!/bin/bash
# pulls latest Hatari sources, builds and copies to known external URL

if [ ! -z "$(/opt/local/bin/pidof automated_build.sh)" ]; then
  exit 0
fi

export SDKROOT="/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.1.sdk"
export PATH="$PATH":/opt/local/bin/

mount -t smbfs //guest@xxxxxxxxxxxxxx/hatari_macos /Users/troed/dev/hatari_macos

cd /Users/troed/dev/hatari
# here to give the network some time to wake up
sleep 5
# if nothing has changed, do nothing
git remote update
git log ..@{u} > ../change.tmp

if [ -s ../change.tmp ]; then
  xcodebuild -sdk "/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.1.sdk" -UseModernBuildSystem=NO clean -project Hatari.xcodeproj > ../hatari_nightlies/build.log

  # has the project regenerated and lost our build script?
  # EXPLANATION: When the Xcode project is re-generated the manually inserted post buildscript is lost. This code re-inserts it. This is a dirty fix that hopefully can be avoided by someone who does this better than me :) This is the script that runs otool and fixes files and paths that are packaged together for a redistributable binary.
  found=`cat "Hatari.xcodeproj/project.pbxproj" | grep script2 | wc -l`
  if [ ${found} -eq 0 ]; then
    a="/* Begin PBXShellScriptBuildPhase section */"
    b=$(cat script2)
    c='/* Build configuration list for PBXNativeTarget "hatari" */;'
    d="                                3B81ADC925A4C8110093582A /* ShellScript */,"
    rm temp
    while IFS= read -r line; do
      input="$line"
      if [[ $line = *"$c" ]]; then
        echo "$input" >> temp
        IFS= read -r line
        echo "$line" >> temp
        IFS= read -r line
        echo "$line" >> temp
        IFS= read -r line
        echo "$line" >> temp
        IFS= read -r line
        echo "$line" >> temp
        IFS= read -r line
        echo "$line" >> temp
        echo "$d" >> temp
      else
        output=${input/$a/"$b"}
        echo "$output" >> temp
      fi
    done < "Hatari.xcodeproj/project.pbxproj"
    mv temp Hatari.xcodeproj/project.pbxproj
  fi
  today=`date '+%Y%m%d_%H%M%S'`
  # EXPLANATION: I have local changes made to a few files that I need to keep.
  git stash
  git checkout master
  git pull
  git stash pop
  ret=$?
  cp ../change.tmp ../change.tm2
# add newline to end of change.log if there is none
# EXPLANATION: Nightly builds include a change.log with commit comments
  [ -n "$(tail -c1 ../hatari_nightlies/change.log)" ] && echo >> ../hatari_nightlies/change.log
  cat ../hatari_nightlies/change.log >> ../change.tm2
  cp ../change.tm2 ../hatari_nightlies/change.log
  if [ ${ret} -eq 0 ]; then
    bsuccess=false

# make sure __DATE__ and __TIME__ are up2date
# EXPLANATION: Doesn't build otherwise
    touch src/main.c
    xcodebuild -sdk "/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.1.sdk" -UseModernBuildSystem=NO -scheme ALL_BUILD -project Hatari.xcodeproj >> ../hatari_nightlies/build.log
    if [ $? -eq 0 ]; then
      # final sanity check that our build script indeed has run
      # EXPLANATION: Dirty hack, counting whether otool has run and now points to included files instead of dynamically linked from the build system
      patched=`otool -L src/Release/Hatari.app/Contents/MacOS/hatari | grep "@executable_path" | wc -l`
      if [ ${patched} -eq 5 ]; then
        bsuccess=true
      fi
    fi

    if [ "$bsuccess" = true ]; then
      # this needs to be hatari in lowercase, CMake generates 'Hatari'
      defaults write /Users/troed/dev/hatari/src/Release/Hatari.app/Contents/Info.plist CFBundleExecutable -string "hatari"

      # EXPLANATION: Well, otherwise we can't codesign ...
      security unlock-keychain -p <password> login.keychain

      export CODESIGN_ALLOCATE="/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/codesign_allocate"
      codesign --force --options=runtime --sign "Developer ID Application: Troed Sangberg (467URBKK99)" src/Release/libFloppy.a >> ../hatari_nightlies/build.log
      codesign --force --deep --options=runtime --sign "Developer ID Application: Troed Sangberg (467URBKK99)" --entitlements /Users/troed/dev/hatari/hatari.app.xcent src/Release/hatari.app >> ../hatari_nightlies/build.log

      dest=Hatari.$today.dmg

      cd /tmp/
      /Users/troed/npm-global/lib/node_modules/appdmg/bin/appdmg.js /Users/troed/dev/hataridev.json $dest >> /Users/troed/dev/hatari_nightlies/build.log

# notarize our dmg
      uuid=$(/usr/bin/xcrun altool --notarize-app --primary-bundle-id "org.tuxfamily.Hatari" --username "troed@xxxxxxxx" --password <password> --file $dest --output-format "xml" | xmllint --xpath "string(//plist/dict/dict/string)" -)

# need to loop here waiting for notarizing to succeed

      count=0
      inprogress=true
      success=false

      while [ "${count}" -lt 20 ] && [ "${inprogress}" == true ]; do
        sleep 60
        result=$(/usr/bin/xcrun altool --notarization-info "$uuid" --username "troed@xxxxxxxx" --password <password> | grep "Status:")
        echo ${result} >> /Users/troed/dev/hatari_nightlies/build.log
        words=($result)
        if [[ "${words[1]}" == "success" ]]; then
          inprogress=false
          success=true
        elif [[ "${words[1]}" == "invalid" ]]; then
          inprogress=false
        else
          ((count++))
        fi
      done

      if [ "$success" = true ]; then
        /usr/bin/xcrun stapler staple $dest

        zip -r $dest.zip $dest
        mv $dest.zip '/Users/troed/dev/hatari_macos/'
        rm -rf /tmp/$dest*
        /Users/troed/dev/notify.sh "Hatari macOS successfully built"
      else
        # signing failed, zip up the log
        dest=Hatari.${today}.build_failed.log
        zip -j ${dest}.zip /Users/troed/dev/hatari_nightlies/build.log
        mv ${dest}.zip '/Users/troed/dev/hatari_macos/'
        /Users/troed/dev/notify.sh "Hatari macOS build failure"
      fi
    else
      # build failed, zip up the log
      dest=Hatari.${today}.build_failed.log
      zip -j ${dest}.zip /Users/troed/dev/hatari_nightlies/build.log
      mv ${dest}.zip '/Users/troed/dev/hatari_macos/'
      /Users/troed/dev/notify.sh "Hatari macOS build failure"
    fi
  else
    # pull failed with unresolved changes
    touch '/Users/troed/dev/hatari_macos/Hatari.'${today}'.build_failed'
    /Users/troed/dev/notify.sh "Hatari macOS merge failure"
  fi
fi

# delete all but the 10 latest .zip files
# see https://stackoverflow.com/questions/25785/delete-all-but-the-most-recent-x-files-in-bash

cd "/Users/troed/dev/hatari_macos/"
ls -tp *.zip | grep -v '/$' | tail -n +11 | tr '\n' '\0' | xargs -0 rm --



------- Original Message -------
On Thursday, April 27th, 2023 at 2:54 AM, Bob Carpenter <hatari@xxxxxxxxxx> wrote:

Thomas,

This build works like the build that Andreas provided. I had to use the “xattr” command from Terminal in order to clear it from quarantine. Your snapshot asked me for permission once to my Documents folder and my Downloads folder. It is missing the other frameworks (PNG, Portmidi, etc.), but this version appears to be working. 

Thanks again for helping us with the Mac version even though you only use Linux.

Bob C


On Apr 26, 2023, at 1:32 PM, Thomas Huth <th.huth@xxxxxxxxx> wrote:

Am Wed, 26 Apr 2023 18:36:47 +0200
schrieb Andreas Grabher <andreas_g86@xxxxxxxxxx>:

I appended a patch that makes it possible to automate the build process quite a bit. It might need some refinement, like only copy frameworks, if they were found and add some additional frameworks as mentioned.

Thanks a lot for the patch ... but I was still hoping that one of the macOS
guys here on the list could come up with a ready patch based on yours that
does all that additional stuff (i.e. take the libraries/frameworks into
account that have been found)... in the current shape, I think it's not
ready for commit yet (e.g. it should still be possible to compile Hatari
without PNG, but currently it's hardcoded there).

These commands were used from the root directory of the sources to create the distributable executable:
#cmake .
#make

Oh, now that's interesting, so you are *not* using the "Xcode" generator
for building macOS release apps? That's surprising, I somehow thought that
"cmake -G Xcode" would be required for building a proper app bundle...?

#install_name_tool -change /opt/local/lib/libz.1.dylib /usr/lib/libz.1.dylib src/hatari.app/Contents/MacOS/hatari
#codesign --force -s - src/hatari.app/Contents/MacOS/hatari

... and this codesign line also looks quite different to the one that the
Xcode generator was trying to execute before I disabled signing there.

Ok, so here's another try, with your patch (partially) applied and with the
standard generator instead of the Xcode one, and with your codesign line
added to the build script:

https://cirrus-ci.com/task/4689992333656064

Chris, Bob, could you please give that build a try, too?

Thanks,
 Thomas







Mail converted by MHonArc 2.6.19+ http://listengine.tuxfamily.org/