Extend z/OS UNIX port to allow building DLL

Updated:
   README.md:  describes process of building/using DLL
   makelibzmq: Build DLL as well as static library (unless BUILD_DLL=false)
   maketests:  Dynamically link to ../src/libzmq.so if present
   runtests:   Explicitly place ../src at start of LIBPATH
   makeclean:  Also remove files created for DLL
   cxxall:     Bumped updated date to reflect last edit

builds/zos/README.md

@@ -12,14 +12,15 @@ Tested build combinations:
 
 * ZeroMQ 4.0.4, using IBM XL C/C++ compiler, as XPLINK in LP64 mode
 
-(Other combinations are likely to work, possibly with minor changes,
-but have not been tested.)
+Other combinations are likely to work, possibly with minor changes,
+but have not been tested.  Both static library and DLL modes have been
+tested.
 
 There are some minor limitations (detailed below), but all core
 functionality tests run successfully.
 
 
-## Quickstart on z/OS UNIX System Services
+## Quickstart: building ZeroMQ on z/OS UNIX System Services
 
 Assuming [z/OS UNIX System
 Services](http://www-03.ibm.com/systems/z/os/zos/features/unix/) is
@@ -36,11 +37,19 @@ installed, ZeroMQ can be built as follows:
 
 *   (Optional) set ZCXXFLAGS for additional compile flags (see below)
 
-*   Build the `libzmq.a` static library with:
+*   Build `libzmq.a` static library and `libzmq.so` dynamic
+    library, with:
 
         cd zeromq-VERSION
         builds/zos/makelibzmq
 
+    or to skip the `libzmq.so` dynamic library:
+
+        cd zeromq-VERSION
+        BUILD_DLL=false
+        export BUILD_DLL
+        builds/zos/makelibzmq
+
 *   (Optional, but recommended) build the core tests with:
 
         builds/zos/maketests
@@ -54,18 +63,61 @@ installed, ZeroMQ can be built as follows:
 
         builds/zos/makeclean
 
+There are details on specifying alternative compilation flags below.
+
+
+## Quickstart: using ZeroMQ on z/OS UNIX System Services
+
+### Static linking
+
+Install `include/*.h` somewhere on your compiler include path.
+
+Install `src/libzmq.a` somewhere on your library search path.
+
+Compile and link application with:
+
+    c++ -Wc,xplink -Wl,xplink ... -+ -o myprog myprog.cpp -lzmq
+
+Run with:
+
+    ./myprog
 
-## Compilation flags
+
+### Dynamic linking
+
+Install `include/*.h` somewhere on your compiler include path.
+
+Install `src/libzmq.so` somewhere on your LIBPATH.
+
+Install `src/libzmq.x` somewhere you an reference for import linking.
+
+Compile and link application:
+
+    c++ -Wc,xplink -Wc,dll ... -+ -c -o myprog.o myprog.cpp
+    c++ -Wl,xplink -o myprog myprog.o /PATH/TO/libzmq.x
+
+Run with:
+
+    LIBPATH=/PATH/OF/LIBZMQ.SO:/lib:/usr/lib:...    # if not in default path
+    ./myprog
+
+
+## Setting other compilation flags
+
+### Optimisation
 
 To build with optimisation:
 
 *   set `ZCXXFLAGS` to "`-O2`" before starting build process above
 
 
+### Full debugging symbols
+
 To build with debugging symbols:
 
 *   set `ZCXXFLAGS` to "`-g`" before starting build process above
 
+### 64-bit mode (LP64/amode=64)
 
 To build in 64-bit mode:
 
@@ -78,6 +130,8 @@ the default for the IBM XL C/C++ compiler.  To build in LP64 mode
 
 (64-bit mode can be combined with optimisation or debug symbols.)
 
+### Combining compilation flags
+
 Other build flags can be used in `ZXCCFLAGS` if desired.  Beware that
 they are passed through (Bourne) shell expansion, and passed to both
 the compile and link stages; some experimentation of argument quoting
@@ -117,14 +171,16 @@ In addition there are some other minor test issues:
     [`libsodium`](http://doc.libsodium.org/), which has not been
     ported to z/OS UNIX System Services yet.
 
-*   `test_monitor` will sometimes fail with `SIGPIPE` (about 1 run
-    in 4); this appears to be a problem with SIGPIPE not being ignored
-    and has been reported upstream.
+*   Some tests will occassionally fail with `SIGPIPE` (about 1 run
+    in 4 one of the tests will fail); this appears to be a problem
+    with SIGPIPE not being ignored and has been reported upstream.
+    The tests work fine if run again.
 
-*   `test_spec_rep` (and possibly other tests) occassionally fail with
-    `Resource temporarily unavailable`, which is a result of EAGAIN
-    not being properly caught in all places and the function call
-    retried.  This has also been reported upstream.
+*   Some tests will occassionally fail with `Resource temporarily
+    unavailable`, which is a result of EAGAIN not being properly
+    caught in all places and the function call retried.  This has
+    also been reported upstream.  Again the tests work fine if
+    run again.
 
 
 ## ZeroMQ on z/OS UNIX System Services: Portability notes

builds/zos/cxxall

@@ -4,7 +4,7 @@
 # additional compile arguments.
 #
 # Written by Ewen McNeill <ewen@imatix.com>, 2014-07-19
-# Updated by Ewen McNeill <ewen@imatix.com>, 2014-07-19
+# Updated by Ewen McNeill <ewen@imatix.com>, 2014-07-22
 #---------------------------------------------------------------------------
 
 VERBOSE="${VERBOSE:-}"    # Set to non-empty for already done status

builds/zos/makeclean

@@ -22,7 +22,7 @@ TESTS="${TOP}/tests"
 
 # Remove object/library files
 echo "Removing libzmq built files"
-(cd "${SRC}" && rm -f *.o *.a *.dbg)
+(cd "${SRC}" && rm -f *.o *.a *.dbg *.x libzmq* *.so)
 
 # Remove test object files
 echo "Removing libzmq tests"

builds/zos/makelibzmq

 #! /bin/sh
-# Build libzmq.a
+# Build libzmq.a static library and libzmq.so dynamic library
+#
+# Usage: makelibzmq
+#        BUILD_DLL=false makelibzmq     # Skip building DLL
+#
+# NOTE: We do a single compile run for both static and dynamic libraries
+# which results in the static library having -Wc,exportall compiled objects;
+# in practice this doesn't seem to cause a problem beyond using some extra
+# space (around 10%).
 #
 # Written by Ewen McNeill <ewen@imatix.com>, 2014-07-21
+# Updated by Ewen McNeill <ewen@imatix.com>, 2014-07-22
 #---------------------------------------------------------------------------
 
 set -e    # Stop on errors
 
+BUILD_DLL="${BUILD_DLL:-true}"     # Build DLL by default
+
 # Figure out where we are
 BIN_DIR=$(dirname $0)
 if [ -z "${BIN_DIR}" ]; then BIN_DIR="."; fi
@@ -14,6 +25,7 @@ case "${BIN_DIR}" in
   /*)                              ;; 
   *)  BIN_DIR="$(pwd)/${BIN_DIR}"; ;;
 esac
+ZCXX="${BIN_DIR}/zc++"
 
 # Locate top of source tree, assuming we're in builds/zos
 TOP="${BIN_DIR}/../.."
@@ -22,9 +34,21 @@ SRC="${TOP}/src"
 # Install pre-generated platform.hpp
 cp -p "${BIN_DIR}/platform.hpp" "${SRC}/platform.hpp"
 
-# Compile all the source
+# Compile all the source (optionally ready for a DLL)
+if [ "${BUILD_DLL}" = "true" ]; then
+  ZCXXFLAGS="${ZCXXFLAGS} -Wc,exportall"
+  export ZCXXFLAGS
+  #echo "Building DLL with ${ZCXXFLAGS}"
+fi
+
 cd "${SRC}"
 "${BIN_DIR}/cxxall"
 
 # Make static library
 ar r libzmq.a *.o
+
+# Optionally Make dynamic library
+if [ "${BUILD_DLL}" = "true" ]; then
+  #echo "Building libzmq.so DLL"
+  "${ZCXX}" -Wl,DLL -o libzmq.so *.o
+fi

builds/zos/maketests

 #! /bin/sh
-# Build tests/* executables; assumes that libzmq.a is already built
+# Build tests/* executables; assumes that libzmq.a or libzmq.so/libzmq.x
+# is already built
+#
+# If libzmq.so and libzmq.x exist, then dynamic linking is used, otherwise
+# static linking is used.
 #
 # Written by Ewen McNeill <ewen@imatix.com>, 2014-07-21
 # Updated by Ewen McNeill <ewen@imatix.com>, 2014-07-22
@@ -27,10 +31,15 @@ TOP="${BIN_DIR}/../.."
 SRC="${TOP}/src"
 TESTS="${TOP}/tests"
 
-if [ -f "${SRC}/platform.hpp" -a -f "${SRC}/libzmq.a" ]; then
-  :
+# Figure out how we are going to link to ZMQ
+LINK_TYPE=unknown
+
+if [ -f "${SRC}/platform.hpp" -a -f "${SRC}/libzmq.so" -a -f "${SRC}/libzmq.x" ]; then
+  LINK_TYPE=dynamic
+elif [ -f "${SRC}/platform.hpp" -a -f "${SRC}/libzmq.a" ]; then
+  LINK_TYPE=static
 else
-  echo "Error: run makezmqlib to build libzmq.a first" >&2
+  echo "Error: run makezmqlib to build libzmq.a and/or libzmq.so/libzmq.x first" >&2
   exit 1
 fi
 
@@ -46,6 +55,12 @@ else
 fi
 
 # Compile all the source
+if [ "${LINK_TYPE}" = "dynamic" ]; then
+  ZCXXFLAGS="${ZCXXFLAGS} -Wc,DLL"
+  export ZXCCFLAGS
+  echo "Building tests to use DLL: ${ZCXXFLAGS}"
+fi
+
 cd "${TESTS}"
 "${BIN_DIR}/cxxall"
 
@@ -58,11 +73,19 @@ skip() {
   fi    
 }
 
-compile() {
+link() {
   OBJ="$1"
   EXE="$2"
   echo " LD ${EXE}"
-  "${ZCXX}" -L ../src -o "${EXE}" "${OBJ}" -lzmq
+  case "${LINK_TYPE}" in
+    static)  "${ZCXX}" -L ../src -o "${EXE}" "${OBJ}" -lzmq
+             ;;
+    dynamic) "${ZCXX}" -o "${EXE}" "${OBJ}" ../src/libzmq.x
+             ;;
+    *)       echo "Do not know how to do ${LINK_TYPE} linking!" 2>&1
+             exit 1
+             ;;
+  esac
 }
 
 for OBJ in *.o; do 
@@ -71,9 +94,9 @@ for OBJ in *.o; do
     if [ "${EXE}" -nt	"${OBJ}" ]; then
       skip "${OBJ}" "${EXE}"
     else
-      compile "${OBJ}" "${EXE}"
+      link "${OBJ}" "${EXE}"
     fi
   else
-    compile "${OBJ}" "${EXE}"
+    link "${OBJ}" "${EXE}"
   fi
 done

builds/zos/runtests

@@ -7,7 +7,7 @@
 # defines the order in which tests are run.
 #
 # Written by Ewen McNeill <ewen@imatix.com>, 2014-07-19
-# Updated by Ewen McNeill <ewen@imatix.com>, 2014-07-21
+# Updated by Ewen McNeill <ewen@imatix.com>, 2014-07-22
 #---------------------------------------------------------------------------
 
 set -e    # Stop if a test fails
@@ -96,6 +96,7 @@ esac
 
 # Locate top of source tree, assuming we're in builds/zos
 TOP="${BIN_DIR}/../.."
+SRCDIR="${TOP}/src"
 TESTDIR="${TOP}/tests"
 
 case "$(pwd)" in
@@ -112,6 +113,11 @@ else
   exit 1
 fi
 
+# Explicitly add SRCDIR into library serach path, to make sure we
+# use our just-built version
+LIBPATH="${SRCDIR}:/lib:/usr/lib"
+export LIBPATH
+
 #---------------------------------------------------------------------------
 # check-TESTS: target from tests/Makefile, converted from Make syntax to
 # shell syntax