WaitLock
WaitLock is a portable UNIX/POSIX command-line tool that provides mutex and semaphore functionality for shell scripts. It enables synchronized access to resources across multiple processes with automatic cleanup when processes die.
Features
Mutex Mode : Single lock holder (default)
: Single lock holder (default) Semaphore Mode : Multiple concurrent lock holders
: Multiple concurrent lock holders Automatic Cleanup : Locks released when process dies
: Locks released when process dies CPU-aware Locking : Can scale locks to CPU count
: Can scale locks to CPU count Lock Inspection : List and check active locks
: List and check active locks Multiple Output Formats : Human, CSV, and null-separated
: Human, CSV, and null-separated Command Execution : Run commands while holding locks
: Run commands while holding locks UNIX Integration : Environment variables, stdin, syslog
: Environment variables, stdin, syslog Portable C Implementation: Runs on any POSIX system
Quick Start
# Install dependencies (Ubuntu/Debian) sudo apt-get install build-essential autoconf # Build and install ./configure make sudo make install # Basic usage - acquire exclusive lock waitlock myapp & # ... do exclusive work ... kill $! # Execute command with lock waitlock database_backup --exec " /usr/local/bin/backup.sh --daily " # List active locks waitlock --list
Table of Contents
Installation
From Source
Prerequisites
C compiler (gcc, clang, or compatible)
GNU Make
autoconf (for building from git)
Build Instructions
# Clone the repository git clone https://github.com/user/waitlock.git cd waitlock # Generate configure script (if building from git) autoreconf -fi # Configure and build ./configure make # Run tests make check # Install system-wide sudo make install # Or install to custom prefix ./configure --prefix=/usr/local make install
Build Options
# Debug build ./configure CFLAGS= " -g -O0 -DDEBUG " # Release build with optimizations ./configure CFLAGS= " -O2 -DNDEBUG " # Cross-compilation example ./configure --host=arm-linux-gnueabihf
Package Installation
# Ubuntu/Debian (when available) sudo apt-get install waitlock # CentOS/RHEL (when available) sudo yum install waitlock # macOS with Homebrew (when available) brew install waitlock
Usage
Basic Syntax
waitlock [options] < descriptor > waitlock --list [--format =< fmt > ] [--all | --stale-only] waitlock --check < descriptor > echo < descriptor > | waitlock [options]
Simple Examples
# Acquire mutex lock waitlock myapp & JOB_PID= $! # ... do exclusive work ... kill $JOB_PID # Check if lock is available if waitlock --check myapp ; then echo " Lock is available " else echo " Lock is held by another process " fi # Execute command while holding lock waitlock backup_job --exec rsync -av /source /destination # Use with timeout waitlock --timeout 30 critical_resource || echo " Timeout! "
Examples
1. Basic Mutex (Exclusive Access)
#! /bin/bash # Ensure only one backup process runs at a time waitlock database_backup || { echo " Another backup is already running " exit 1 } # Perform backup mysqldump --all-databases > backup.sql gzip backup.sql # Lock automatically released when script exits
2. Semaphore (Multiple Concurrent Access)
#! /bin/bash # Allow up to 4 concurrent download processes waitlock --allowMultiple 4 download_pool || { echo " Too many downloads already running " exit 1 } # Perform download wget " https://example.com/file.tar.gz " # Lock automatically released when script exits
3. CPU-Based Semaphore
#! /bin/bash # Use one lock per CPU core, reserving 2 cores for system waitlock --onePerCPU --excludeCPUs 2 cpu_intensive_task || { echo " All CPU slots are busy " exit 1 } # Run CPU-intensive task ./compute_job.sh
4. Command Execution Mode
#! /bin/bash # Execute command while holding lock (recommended approach) waitlock database_backup --exec bash -c " mysqldump --all-databases > backup.sql gzip backup.sql echo 'Backup completed' "
5. Lock Monitoring and Management
#! /bin/bash # Monitor active locks # List all locks in human-readable format waitlock --list # List in CSV format for parsing waitlock --list --format csv # Show only stale locks waitlock --list --stale-only # Count active locks waitlock --list --format csv | tail -n +2 | wc -l
6. Pipeline and Batch Processing
#! /bin/bash # Process files with controlled parallelism find /data -name " *.csv " | while read file ; do basename " $file " | waitlock --allowMultiple 3 --exec process_file " $file " done # Or with xargs for better performance find /data -name " *.csv " | \ xargs -P 10 -I {} sh -c ' waitlock -m 3 batch_processor --exec "process_file {}" '
7. Using with Environment Variables
#! /bin/bash # Configure via environment variables export WAITLOCK_TIMEOUT=60 export WAITLOCK_DIR= " /var/lock/myapp " export WAITLOCK_DEBUG=1 waitlock myapp_task --syslog --syslog-facility local0
8. Error Handling
#! /bin/bash # Robust error handling waitlock --timeout 30 critical_resource case $? in 0) echo " Lock acquired successfully " ;; 1) echo " Lock is busy " >&2 ; exit 1 ;; 2) echo " Timeout expired " >&2 ; exit 1 ;; 3) echo " Usage error " >&2 ; exit 1 ;; * ) echo " Unexpected error " >&2 ; exit 1 ;; esac # Your critical section here perform_critical_operation
9. Resource Pool Management
#! /bin/bash # Manage GPU resources # Export slot number for GPU selection waitlock --allowMultiple 4 gpu_pool & LOCK_PID= $! # Wait for lock and get slot number wait $LOCK_PID if [ $? -eq 0 ] ; then # Use WAITLOCK_SLOT environment variable export CUDA_VISIBLE_DEVICES= $WAITLOCK_SLOT ./gpu_computation.py fi
10. Distributed Locking (NFS)
#! /bin/bash # Coordinate across multiple machines using NFS export WAITLOCK_DIR= " /mnt/shared/locks " waitlock cluster_job --timeout 300 --exec bash -c " echo 'Running on $( hostname ) ' ./distributed_task.sh "
Command Reference
Core Options
Option Description -m, --allowMultiple N Allow N concurrent holders (semaphore mode) -c, --onePerCPU Allow one lock per CPU core -x, --excludeCPUs N Reserve N CPUs (reduce available locks by N) -t, --timeout SECS Maximum wait time before giving up --check Test if lock is available without acquiring -e, --exec CMD Execute command while holding lock
Output Options
Option Description -q, --quiet Suppress all non-error output -v, --verbose Verbose output for debugging -f, --format FMT Output format: human, csv, null --syslog Log operations to syslog --syslog-facility FAC Syslog facility (daemon|local0-7)
Management Options
Option Description -l, --list List active locks and exit -a, --all Include stale locks in list --stale-only Show only stale locks
Configuration Options
Option Description -d, --lock-dir DIR Directory for lock files -h, --help Show usage information -V, --version Show version information
Environment Variables
Variable Description Default WAITLOCK_DIR Lock directory path auto-detect WAITLOCK_TIMEOUT Default timeout in seconds infinite WAITLOCK_DEBUG Enable debug output disabled WAITLOCK_SLOT Preferred semaphore slot auto
Environment Variable Examples
# Set default timeout export WAITLOCK_TIMEOUT=300 # Use custom lock directory export WAITLOCK_DIR= " /var/lock/myapp " # Enable debug output export WAITLOCK_DEBUG=1 # Prefer specific semaphore slot export WAITLOCK_SLOT=2
Exit Codes
Code Meaning 0 Success 1 Lock is busy 2 Timeout expired 3 Usage error 4 System error 5 Permission denied 6 Lock directory not accessible 75 Temporary failure 126 Command not executable 127 Command not found
Advanced Usage
Syslog Integration
# Log all operations to syslog waitlock --syslog --syslog-facility local0 myapp # Monitor syslog for lock operations tail -f /var/log/syslog | grep waitlock
Lock File Format
WaitLock uses binary lock files with the following structure:
Magic number (0x57414C4B = "WALK")
Process metadata (PID, PPID, UID)
Lock information (type, slot, max holders)
Timestamps and command line
CRC32 checksum for integrity
Platform Support
WaitLock is tested on:
Linux (glibc, musl)
FreeBSD
OpenBSD
NetBSD
macOS
Performance Considerations
Lock files are stored in /var/lock/waitlock (system) or /tmp/waitlock (user)
(system) or (user) Directory scanning is O(n) where n = number of lock files
Use hierarchical descriptors for namespace separation
Consider tmpfs for high-frequency locking
Troubleshooting
Common Issues
Permission Denied # Check directory permissions ls -la /var/lock/waitlock # Use user-specific directory export WAITLOCK_DIR= " $HOME /.waitlock " Stale Locks # List stale locks waitlock --list --stale-only # Clean up automatically (locks are cleaned on next access) waitlock --check any_descriptor High Contention # Monitor lock contention waitlock --verbose --timeout 1 busy_resource # Use exponential backoff (built-in) waitlock --timeout 60 busy_resource
Debug Mode
# Enable debug output export WAITLOCK_DEBUG=1 waitlock --verbose myapp # Or use command line waitlock --verbose myapp
Contributing
Development Setup
# Clone repository git clone https://github.com/user/waitlock.git cd waitlock # Install development dependencies sudo apt-get install autoconf automake libtool # Generate build files autoreconf -fi # Configure for development ./configure --enable-debug CFLAGS= " -g -O0 " # Build and test make make check
Running Tests
# Run internal test suite ./src/waitlock --test # Run shell-based tests ./test_basic.sh ./test_semaphore.sh ./test_timeout.sh
Code Style
Follow POSIX C89/C90 standards
Use 4-space indentation
Include comprehensive error handling
Add tests for new features
Submitting Changes
Fork the repository Create a feature branch Make changes with tests Submit a pull request
License
WaitLock is released under the MIT License. See LICENSE for details.
Support
Documentation : See man waitlock after installation
: See after installation Issues : Report bugs on GitHub Issues
: Report bugs on GitHub Issues Discussions: Join discussions on GitHub Discussions
Acknowledgments