1 Topic by mfleetwo

  • mfleetwo
  • Developer
  • Offline
  • Registered: 2012-05-18
  • Posts: 477

Topic: Test compiling every commit in a patch set using git-test-sequence

I have found git-test-sequence (blog: git test-sequence: Push Working Changes) which makes test compiling all the commits in a patch set easy.  It's only a small changed to the work flow we currently recommend to developers to check their patch set before submission.

cd /tmp
git clone git://git.gnome.org/gparted gparted-test
cd gparted-test
cp -p git-test-sequence testbuild.sh ~/bin
git am ~/mypatch.mbox
git-test-sequence origin/master.. testbuild.sh


Here's a worked example showing several test builds:

$ cd /tmp
$ git clone git://git.gnome.org/gparted gparted-test
Cloning into gparted-test...
remote: Counting objects: 12699, done.
remote: Compressing objects: 100% (6123/6123), done.
remote: Total 12699 (delta 10191), reused 7819 (delta 6525)
Receiving objects: 100% (12699/12699), 5.10 MiB | 390 KiB/s, done.
Resolving deltas: 100% (10191/10191), done.
$ cd gparted-test

Use a couple of my in progress patches for testing.

$ git am ~/mypatches.mbox
Applying: Allow ntfsresize command to report real-time %age progress (#697662)
Applying: Correctly report usage of absolutely full NTFS (#697946)

Here's the top few commits so you can see them and where the git tags point, especially "origin/master" (master tag in the upstream repository at gnome.org).

$ git log --decorate --pretty=oneline -3
a6ebcbc34ec14b4cf5235e944df9cbdbae575173 (HEAD, master) Correctly report usage of absolutely full NTFS (#697946)
0d6f7343e8f78c527ddced1972c9b50510a90062 Allow ntfsresize command to report real-time %age progress (#697662)
76a9cd3c931adc92b49a9effc61ddbfeccb9dd07 (origin/master, origin/HEAD) Updated Spanish translation

Run testbuild.sh over a range of git commits specified using standard commit ranges syntax  (about 2/3 down the page).

$ git-test-sequence origin/master.. testbuild.sh
Testing testbuild.sh -j 8
HEAD is now at 0d6f734 Allow ntfsresize command to report real-time %age progress (#697662)
Passed: 40912c57d5330bd3a68b80b855e5a55178b96ca1.
Testing testbuild.sh -j 8
HEAD is now at a6ebcbc Correctly report usage of absolutely full NTFS (#697946)
Passed: ab1090f871ccb74d1679af4c14ea1d9aeec10672.
All's well.
$ echo $?
0

Re-run the tests again.  Results are returned immediately, having been cached in the local git object store.

$ git-test-sequence origin/master.. testbuild.sh
Already pass 40912c57d5330bd3a68b80b855e5a55178b96ca1
Already pass ab1090f871ccb74d1679af4c14ea1d9aeec10672
All's well.
$ echo $?
0

Test build GParted over a different range of commits to show a failure case.  Stops at the first commit which fails to build.

$ rm testbuild.log
$ git-test-sequence GPARTED_0_14_1..GPARTED_0_15_0 testbuild.sh
Testing testbuild.sh
HEAD is now at c9e6dca Append -git to version for continuing development
Passed: 261d9602fc3b47bb4495c2b290e67c01868e21f4.
...
Testing testbuild.sh
HEAD is now at 34da790 Fix move partition right to left shrinks partition 1 MiB (#695078)
Passed: 528b0ddeec2ee50b89a247f6d625524c9e648ed1.
Testing testbuild.sh
HEAD is now at 08924d3 Updated slovak translation
Passed: 92883b27e2d1a3e13160db573c728b69853a4140.
Testing testbuild.sh
HEAD is now at f5a5c9c Remove gdk_threads_enter/exit (#685740)
Broke on f5a5c9cdb9d24c930871060bceade9cfe0e69053
You have new mail in /var/spool/mail/mike
$ echo $?
1

Look in the testbuild.log for the build results.

$ tail -18 testbuild.log 
g++ -DHAVE_CONFIG_H -I. -I.. -pthread -I/usr/include/glib-2.0 -I/usr/lib/glib-2.0/include   -pthread -I/usr/include/gtkmm-2.4 -I/usr/lib/gtkmm-2.4/include -I/usr/include/atkmm-1.6 -I/usr/include/giomm-2.4 -I/usr/lib/giomm-2.4/include -I/usr/include/pangomm-1.4 -I/usr/lib/pangomm-1.4/include -I/usr/include/gtk-2.0 -I/usr/include/gtk-unix-print-2.0 -I/usr/include/gdkmm-2.4 -I/usr/lib/gdkmm-2.4/include -I/usr/include/atk-1.0 -I/usr/include/glibmm-2.4 -I/usr/lib/glibmm-2.4/include -I/usr/include/glib-2.0 -I/usr/lib/glib-2.0/include -I/usr/include/sigc++-2.0 -I/usr/lib/sigc++-2.0/include -I/usr/include/cairomm-1.0 -I/usr/lib/cairomm-1.0/include -I/usr/include/pango-1.0 -I/usr/include/cairo -I/usr/include/pixman-1 -I/usr/include/freetype2 -I/usr/include/libpng12 -I/usr/lib/gtk-2.0/include -I/usr/include/gdk-pixbuf-2.0   -DGPARTED_DATADIR=\""/usr/local/share"\" -DGNOMELOCALEDIR=\""/usr/local/share/locale"\"   -Wall        -g -O2 -MT GParted_Core.o -MD -MP -MF .deps/GParted_Core.Tpo -c -o GParted_Core.o GParted_Core.cc
GParted_Core.cc:3551:2: error: ‘Threads’ in namespace ‘Glib’ does not name a type
GParted_Core.cc:3552:2: error: ‘Threads’ in namespace ‘Glib’ does not name a type
GParted_Core.cc: In function ‘bool GParted::_ped_exception_handler(GParted::ped_exception_ctx*)’:
GParted_Core.cc:3571:8: error: ‘struct GParted::ped_exception_ctx’ has no member named ‘cond’
GParted_Core.cc:3579:7: error: ‘struct GParted::ped_exception_ctx’ has no member named ‘mutex’
GParted_Core.cc:3580:7: error: ‘struct GParted::ped_exception_ctx’ has no member named ‘cond’
GParted_Core.cc:3581:7: error: ‘struct GParted::ped_exception_ctx’ has no member named ‘mutex’
GParted_Core.cc: In static member function ‘static PedExceptionOption GParted::GParted_Core::ped_exception_handler(PedException*)’:
GParted_Core.cc:3591:7: error: ‘struct GParted::ped_exception_ctx’ has no member named ‘mutex’
GParted_Core.cc:3593:7: error: ‘struct GParted::ped_exception_ctx’ has no member named ‘cond’
GParted_Core.cc:3593:22: error: ‘struct GParted::ped_exception_ctx’ has no member named ‘mutex’
GParted_Core.cc:3594:7: error: ‘struct GParted::ped_exception_ctx’ has no member named ‘mutex’
make[2]: *** [GParted_Core.o] Error 1
make[2]: Leaving directory `/tmp/gparted-test/src'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/tmp/gparted-test'
make: *** [all] Error 2


Testbuild.sh script looks like this:

#!/bin/sh
# Build GParted logging the top commit and build results.

exec 1>> testbuild.log 2>&1
echo '############################################################'
echo "## Build date: `date`"
echo '############################################################'
git log -1
echo '############################################################'

set -x

if [ $# -eq 0 ]; then
    # Default to using the number of processors to tell make how
    # many jobs to run simultaneously
    nproc=`grep -c '^processor' /proc/cpuinfo`
    set -- -j $nproc
fi

# Disable bold text escape sequences from gnome-autogen.sh
TERM=

./autogen.sh && make clean && make "${@}"


Curtis,

I'm proposing the following:

  • Add git-test-sequence and testbuild.sh to the GParted code.

  • We update the GParted GIT page probably with a "How to review patch sets" section.

If this sounds OK, I'll produce a patch and draft some wording.  This will allow us to easily check all patches compile.

Thanks,
Mike

2 Reply by gedakc

  • gedakc
  • Developer
  • Offline
  • Registered: 2007-11-30
  • Posts: 4,593

Re: Test compiling every commit in a patch set using git-test-sequence

Hi Mike,

This sounds like a great idea!

Please do proceed with this proposal.  When ready I can update the GParted git page accordingly.

Regards,
Curtis

3 Reply by mfleetwo

  • mfleetwo
  • Developer
  • Offline
  • Registered: 2012-05-18
  • Posts: 477

Re: Test compiling every commit in a patch set using git-test-sequence

Tested build speed on a range of systems varying the number of simultaneous jobs make is told to use.  Build speed is maximised by using the number of processors as the number of simultaneous make jobs.  Using more jobs doesn't make the build any faster.

Test on:

  • Physical 4 core - (4 jobs)

  • Physical single core with hyper-threading - (2 jobs)

  • VirtualBox single core - (1 job)

  • VirtualBox dual core - (2 jobs)

Updated the initial post thus:

  • Removed "-j #" from the testbuild.sh command line everywhere

  • Updated testbuild.sh script to set the number of simultaneous make jobs.  Overridable on the command line

4 Reply by gedakc

  • gedakc
  • Developer
  • Offline
  • Registered: 2007-11-30
  • Posts: 4,593

Re: Test compiling every commit in a patch set using git-test-sequence

'Good to know where the optimum values are as this will help save time over the long haul.

5 Reply by mfleetwo

  • mfleetwo
  • Developer
  • Offline
  • Registered: 2012-05-18
  • Posts: 477

Re: Test compiling every commit in a patch set using git-test-sequence

Hi Curtis,

Bug 699881 - testbuild.sh - Builds GParted logging results has been opened to add the script to GParted.

Following posts will be:

  1. New draft "Verifying every change of a patch set compiles" section for the GParted Git page

  2. Notes on testbuild.sh

  3. Notes on git-test-sequence

Then a final "I've finished editing the posts.  OK to use post".

Thanks,
Mike

6 Reply by mfleetwo

  • mfleetwo
  • Developer
  • Offline
  • Registered: 2012-05-18
  • Posts: 477

Re: Test compiling every commit in a patch set using git-test-sequence

Verifying every change of a patch set compiles

NOTE: Ensuring that every change in a multi patch set compiles is important for incremental testability and enabling the use of git bisect in searching for the commit which introduced a bug.


1. One time set-up of git-test-sequence and testbuild.sh:

mkdir -p ~/bin
cd ~/bin
wget https://raw.github.com/dustin/bindir/master/git-test-sequence
chmod 755 git-test-sequence
wget https://git.gnome.org/browse/gparted/plain/testbuild.sh
chmod 755 testbuild.sh

2. Clone GParted for testing:

cd /tmp
git clone git://git.gnome.org/gparted gparted-test
cd gparted-test

3. Apply and test build every patch in the set:

git am ~/review-patchset.mbox
git-test-sequence origin/master.. testbuild.sh

Further information can be found in the GParted forum about testbuild.sh and git-test-sequence.

7 Reply by mfleetwo

  • mfleetwo
  • Developer
  • Offline
  • Registered: 2012-05-18
  • Posts: 477

Re: Test compiling every commit in a patch set using git-test-sequence

testbuild.sh

testbuild.sh is for developers to build GParted in a git repository, appending the top commit and build results to the log file testbuild.log.  It is intended for use with git-test-sequence to verify every commit in a patch set compiles, but it can be used standalone too.


Build current code:

rm testbuild.log
testbuild.sh
echo $?
less testbuild.log

Parameters on the command line are passed to autogen.sh, which in turn are passed on to configure.  See the INSTALL file for details on parameters to configure.  Example:

testbuild.sh --prefix=/usr

By default testbuild.sh instructs make to use the number of processors in the machine as the number of jobs to run simultaneously.  This is to minimise build time.  This can be overridden by specifying the required parameters to make in the MAKEFLAGS environment variable.  See make(1) for details of parameters to make.  Example:

MAKEFLAGS='-j 2 -w' testbuild.sh

Build current code specifying both alternative make flags and configure parameters:

rm testbuild.log
MAKEFLAGS='-j 2 -w' testbuild.sh --prefix=/usr
echo $?
less testbuild.log

8 Reply by mfleetwo

  • mfleetwo
  • Developer
  • Offline
  • Registered: 2012-05-18
  • Posts: 477

Re: Test compiling every commit in a patch set using git-test-sequence

git-test-sequence

git-test-sequence (blog: git test-sequence: Push Working Changes) allows a sequence of git commits to be tested.  The range of commits is specified using standard git commit ranges syntax (about 2/3 down the page).  Combined with testbuild.sh it allows us to verify every change of a patch set compiles.


Git-test-sequence records test results in the local git repository object store allowing it to report immediately when passed tests are re-run again.


Usage:

git-test-sequence <commit_range> <test_program>

Normal case would be to apply patch set for review and test build all commits between upstream master and current head:

cd /tmp
git clone git://git.gnome.org/gparted gparted-test
cd gparted-test
git am ~/review-patchset.mbox
git-test-sequence origin/master.. testbuild.sh

Test build a range of commits while also specifying parameters to configure and alternative make flags:

cd /tmp
git clone git://git.gnome.org/gparted gparted-test
cd gparted-test
MAKEFLAGS='-j 2 -w' git-test-sequence GPARTED_0_14_1..GPARTED_0_15_0 'testbuild.sh --prefix=/usr'

On failure examine the log file testbuild.log for details.

9 Reply by mfleetwo

  • mfleetwo
  • Developer
  • Offline
  • Registered: 2012-05-18
  • Posts: 477

Re: Test compiling every commit in a patch set using git-test-sequence

OK Curtis,

I've finished editing for now.  You can copy the "Verifying every change of a patch set compiles" to the GParted GIT page.  It contains a link to the testbuild.sh script in the public GParted git repository, which obviously doesn't exist until Bug 699881 is committed.

I may update the "git-test-sequence" post later if more information is required.

Thanks,
Mike

10 Reply by gedakc

  • gedakc
  • Developer
  • Offline
  • Registered: 2007-11-30
  • Posts: 4,593

Re: Test compiling every commit in a patch set using git-test-sequence

Hi Mike,

The sections "Verifying every change of a patch set compiles", "testbuild.sh", and "git-test-sequence" have been duplicated on the GParted git web page.  If you notice any errors or desire changes, then just let me know.

Thanks again for sharing this test build discovery with me and other GParted developers.  :-)

Curtis