Andre Noll [Sun, 28 Apr 2024 03:10:19 +0000 (05:10 +0200)]
Avoid rebuilds of lopsub-generated files.
Without having the .PRECIOUS special target depending on these, the
files get removed after a successful build and a subsequent make run
has to rebuild them.
Andre Noll [Sun, 28 Apr 2024 01:25:28 +0000 (03:25 +0200)]
Make the build quiet by default.
This teaches make(1) to give all lines of the recipe to a single
shell invocation, speeding up the build and facilitating the simple
SAY function to quieten the build. The old verbose output can still
be obtained by running make V=1.
Andre Noll [Sat, 27 Apr 2024 22:35:06 +0000 (00:35 +0200)]
build: Dynamic dependency creation and versioning.
This gets rid of Makefile.deps in favor of one auto-generated .d
file per .c file, and adds version-gen.sh, a shell script executed
by make(1) which tries to determine the version number that is going
to be incorporated into the executable via the generated version.c.
Andre Noll [Sun, 12 May 2024 01:09:58 +0000 (03:09 +0200)]
Consult $HOME and $LOGNAME rather than calling getpwuid(),
Quiting getpwnam(3):
An application that wants to determine its user’s home
directory should inspect the value of HOME (rather than the value
getpwuid(getuid())->pw_dir) since this allows the user to modify
their notion of "the home directory" during a login session.
The old behaviour of get_homedir() to return "/tmp" in the error case
is misleading at best, so abort in this case.
Andre Noll [Sat, 27 Apr 2024 21:45:59 +0000 (23:45 +0200)]
Resolve config file path only once.
Currently we call get_config_file_name() before each call call to a
public function of ipc.c because those functions compute the IPC key
from the filename. This is unnecessary and caused several bugs. Clean
up this mess by computing the filename only once and store it in a
global variable.
Andre Noll [Thu, 25 Apr 2024 13:38:23 +0000 (15:38 +0200)]
Support local make files.
These additional targets are handy for site-local targets such as
installing the web files or deploying the executable and the man page
on a remote server.
Andre Noll [Tue, 19 Sep 2023 14:31:49 +0000 (16:31 +0200)]
Fix --config-file for relative paths.
The dss lock works by first turning the given config file path
argument into a canonical absolute path using dss_realpath(), then
hashing this absolute path to obtain a key ID for semget(2).
If the given path is relative, we have to compute the ID before
changing to the destination directory because dss_realpath() needs to
call stat(2) to detect symlinks, and this system call will fail if the
current working directory has changed. This is currently not the case
as we change to the destination directory early in check_config().
If dss_realpath() fails, we silently use the unmodified path argument
for hashing to deal with the case that the default config does not
exist. As a result, if relative paths are given, the key ID depends
on whether or not change_to_dest_dir() was called. This is the case
for the run subcommanmd, but not for the kill subcommand. Thus the
kill subcommand does not work as expected if a relative path is given.
Fix this by grabbing the lock before changing the working directory
in all cases.
Andre Noll [Sun, 17 May 2020 15:08:33 +0000 (17:08 +0200)]
Avoid duplicate error message.
If parse_config_file() encounters an error, it logs the error *and*
returns the error code, which causes the error message to be logged
again. Fix this by removing the log statement from parse_config_file().
Andre Noll [Thu, 7 Nov 2019 11:31:02 +0000 (12:31 +0100)]
Merge branch 'refs/heads/t/prune'
A couple of cleanups and unifications for the snapshot pruning code,
which is executed by the prune and run subcommands. With the patches
applied, both subcommands behave identically, i.e. they consider
the same set of snapshots as candidates for pruning. Also the prune
command gained the new --disk-space option to force it to act as if
disk space was high or low.
Cooking for six weeks.
* refs/heads/t/prune:
Introduce prune --disk-space.
Revamp com_prune().
Factor out find_removable_snapshot().
prune Simplify rm exit code logic.
find_oldest_removable_snapshot(): Improve log message.
prune: Print a message if there is nothing to prune.
get_snapshot_list(): Add comment about sorting.
prune: Fail gracefully if pre-rm hook fails.
Andre Noll [Fri, 25 Oct 2019 11:34:49 +0000 (13:34 +0200)]
daemon_init(): Do not set umask to zero.
We don't need this because we actively want rsync to respect the file
creation mask. This does not matter in the common case of a single
source directory because the permission bits will be copied from the
source directory anyway. However, for multiple source directories it
does matter because with umask(0) the top level snapshot directory
is created with mode 777.
Andre Noll [Thu, 7 Feb 2019 22:05:34 +0000 (23:05 +0100)]
Introduce prune --disk-space.
The new option takes a mandatory argument of the set {check, high,
low}. The latter two make the command act as if disk space was
high/low without actually performing the check. This is mostly useful
for debugging.
Andre Noll [Sun, 1 Sep 2019 17:34:54 +0000 (19:34 +0200)]
Revamp com_prune().
The prune subcommand implements its own logic for picking the snapshot
to remove. The algorithm is similar but not identical to how the run
subcommand gets rid of snapshots.
This patch eliminates this inconsistency by changing com_prune()
to call the find_removable_snapshot() helper which was introduced in
the previous commit.
Since find_removable_snapshot() returns a dynamically allocated string
via the "why" pointer, we have to introduce another label for freeing
this memory.
The patch also improves the help text of the prune subcommand slightly.
Andre Noll [Thu, 7 Feb 2019 19:49:19 +0000 (20:49 +0100)]
Factor out find_removable_snapshot().
Both the run and the prune subcommand contain code for picking a
suitable snapshot to remove, with slightly different semantics.
This is a preparatory patch for eliminating the differences between
the two implementations. It introduces a new helper which can be
shared. After this patch, only the run subcommand uses the helper. A
subsequent patch will convert the second caller, com_prune().
The other three functions which return a candidate for removal print
one debug message but not the name of the selected snapshot, so make
find_oldest_removable_snapshot() follow this pattern too.
Andre Noll [Thu, 7 Feb 2019 18:05:24 +0000 (19:05 +0100)]
prune: Print a message if there is nothing to prune.
When the prune subcommand finds there are no snapshots to prune, the
command stays silent, which is confusing. Make it print "nothing to
prune" in this case.
Andre Noll [Sun, 1 Sep 2019 17:29:39 +0000 (19:29 +0200)]
prune: Fail gracefully if pre-rm hook fails.
In this case wait_for_remove_process() returns non-negative and we
miss to set the exit code, making the command appear to succeed even
if the rm process has not been created.
Andre Noll [Sat, 19 Jan 2019 11:16:24 +0000 (12:16 +0100)]
Redo the logo as svg.
Scalable vector graphics are much nicer for web pages than bitmaps. The
mklogo script can be removed and there is no more dependency on
ImageMagick. What a deal.
The new logo was made with vi. It is a bit smaller than the old logo
but looks similar.
In the calculation of the length of the argv array we did not take
into account that --source-dir may be given multiple times. This can
result in an invalid write at the end of the allocated space.
Andre Noll [Thu, 7 Feb 2019 20:41:22 +0000 (21:41 +0100)]
Remove stale comment.
This comment went stale ten years ago in commit 360bcc95d588 (Clean
up snapshot removal logic) which changed the type of the return value
of find_redundant_snapshot() from int to struct snapshot *.
Andre Noll [Wed, 6 Jun 2018 13:08:24 +0000 (15:08 +0200)]
Support multiple source directories.
rsync is capable of copying multiple source directories to a single
destination, but this is currently not supported by dss. This commit
adds this functionality. The implementation is straight-forward,
except that we don't want to add a trailing slash to every source
directory. The new comment in dss.c explains this in more detail.
Andre Noll [Tue, 14 Nov 2017 03:12:02 +0000 (04:12 +0100)]
run: Don't kill children twice.
When handle_signal(), the signal dispatcher of the run subcommand,
detects that SIGINT or SIGTERM was received, it calls kill_children()
to terminate any running rsync or rm processes. It then returns
negative which terminates the select loop. However, after select_loop()
returns, kill_children() is called again. Also the error message is
logged twice.
Not a biggie, but let's get rid of this redundancy by removing the
first call to kill_children().
Since handle_signal() is only called from com_run(), this patch
affects only the run subcommand.
Andre Noll [Tue, 14 Nov 2017 02:19:58 +0000 (03:19 +0100)]
Fix compute_next_snapshot_time().
The function computes the average idle time between snapshots and adds
this value to the completion time of the last snapshot to obtain the
start time for the next snapshot.
However, if the last snapshot happens to be incomplete, its completion
time is set to -1. Hence the computed next snapshot time is going to
be in the past, so we start the next snapshot immediately.
Although this is incorrect, the bug is benign because the correct next
snapshot time should also be in the past since we decided earlier to
create the snapshot which was now found incomplete.
Fix this by using the completion time of the last _complete_ snapshot
instead.
Andre Noll [Tue, 14 Nov 2017 02:14:47 +0000 (03:14 +0100)]
Silence a bogus scan-build warning.
We never pass a NULL pointer to create_snapshot(), but scan-build
is unable to prove this and claims that the array access results in
a null pointer dereference. The added assertion helps the reader of
the code, and it quietens scan-build.
Andre Noll [Mon, 6 Nov 2017 00:12:41 +0000 (01:12 +0100)]
Replace license boilerplate with single line SPDX comments.
This gets rid of existing copyright templates in favor of just the
one-liner SPDX (Software Package Data Exchange) notice. All files are
licensed under the GPL-2.0, so the same tag is added to each file. No
copyright is changed by this commit.
Several files (mostly the very short ones) did not contain a license
text so far. By default all files without license information are
under the default license of this package, which is GPL version 2.
This commit adds the missing SPDX line so that now all files except
dss.css, index.html.in, INSTALL, NEWS and README have it.
We also remove author and copyright year, since the author is the same
everywhere, and the year hasn't been updated any more since at least
six years. Accurate information is available from the git log.
The COPYING file can also be removed because the license text at
https://spdx.org/licenses/GPL-2.0.html is immutable.
Andre Noll [Sat, 11 Nov 2017 05:24:47 +0000 (06:24 +0100)]
find_orphaned_snapshot(): Improve log message.
It is kind of obvious that find_orphaned_snapshot() looks for, well,
orphaned snapshots. The new message at least gives the user a vague
idea what this means.
Andre Noll [Mon, 6 Nov 2017 00:52:03 +0000 (01:52 +0100)]
gcc-compat.h: Remove _static_inline_ macro.
The only purpose of this macro is to have a way to include static
inline functions into the doxygen source code documentation (but omit
normal static functions in .c files). Since dss does not use doxygen,
the macro is pointless.
Remove the equally pointless documentation of dss_rename(), one
of the two users of _static_inline_, while converting it to plain
static inline.
Andre Noll [Mon, 13 Nov 2017 16:46:57 +0000 (17:46 +0100)]
Merge branch 'refs/heads/t/configtest'
A single patch which adds the new configtest subcommand, plus a fixup
for a formatting issue which was noticed only after the branch had
already been merged to next.
* refs/heads/t/configtest:
show_subcommand_summary(): Increase column width.
New subcommand: configtest.
Andre Noll [Sun, 15 Oct 2017 16:08:47 +0000 (18:08 +0200)]
kill: New option --wait.
Simply running "dss kill" during system shutdown to terminate the
dss process does not work as expected because the kill subcommand
exits after the signal has been sent, which might be long before the
targeted dss process terminates.
For example, the dss main process might be running its exit hook to
inform the system administrator about the fact that the dss service
is going down when the shutdown procedure already has deactivated the
network. Or the shutdown procedure kills the exit hook with SIGKILL
during its normal "killing remaining processes" phase before file
systems are unmounted.
With the --wait option, the kill subcommand will not return until the
dss process has died or the timeout expires. We hardcode the timeout
in send_signal() for the time being. It can be made configurable if
this turns out to be necessary.
Andre Noll [Sun, 15 Oct 2017 18:29:39 +0000 (20:29 +0200)]
Allow word-splitting for exit hook.
All hooks except the exit hook are run via dss_exec_cmdline_pid(),
which performs word splitting to create the argument vector for
exec(2). For the exit hook, however, we build the argument vector
manually, so the command line for the exit hook is not split.
This commit removes this inconsistency. However, we can't use
dss_exec_cmdline_pid() here because we need to append the error string
which caused dss to exit to the argument vector as a single argument,
and this string may well contain whitespace characters.
Hence we run split_args() on the argument to --exit-hook to obtain
an argument vector, append the error string as another element,
and then run dss_exec().
Andre Noll [Mon, 6 Nov 2017 02:34:39 +0000 (03:34 +0100)]
README/INSTALL: Fix typo: snaphot.
The typo in README was introduced two years ago in commit 05e75054
(README: Explain that there are no incremental backups) while the
one in INSTALL is more than five years old, see commit a0b6810b (doc:
Add a second example config file).
Andre Noll [Sun, 15 Oct 2017 19:11:46 +0000 (21:11 +0200)]
ls: Print current duration of incomplete snapshots.
Currently the duration of incomplete (and orphaned) snapshots is shown
as 0:00. It's more interesting to see for how long the snapshot is
already being created, so print the difference of the current time
and the start time instead.
Fix an overlong line and a whitespace issue while at it.
Andre Noll [Mon, 16 Oct 2017 15:36:51 +0000 (17:36 +0200)]
run: Wait for children to die.
When the select loop returns and dss is about to terminate, it sends
SIGTERM to any running rm or rsync processes and exits. It does not
wait for these processes to die, however. This is trivial to implement,
and it makes life easier for shutdown scripts which like to proceed
with unmounting file systems.
Andre Noll [Sun, 15 Oct 2017 13:03:08 +0000 (15:03 +0200)]
New subcommand: configtest.
Similar to the identically named subcommand of apache2ctl. This is
trivial to implement because we only need to describe the subcommand
in dss.suite and create a command handler which prints an OK message
and returns success. If the config file contains errors, we abort
earlier anyway.
Andre Noll [Tue, 17 Oct 2017 17:11:07 +0000 (19:11 +0200)]
Save the subcommand pointer in a global variable.
This is needed for subcommand sensitive logging which will be
introduced in a subsequent commit. For now it allows to drop the
argument of check_config(), which is good given that handle_sighup()
already played dirty games by "knowing" it is only called from the
run subcommand.
Andre Noll [Mon, 16 Oct 2017 15:19:08 +0000 (17:19 +0200)]
New option: --mountpoint.
The new option applies to run, create, ls and prune. The feature
could be implemented as a pre-create hook, but since it is so common,
it makes sense to add it to dss proper.
As for the implementation we simply check that "." and ".." are on
different devices (or are identical).
Andre Noll [Sun, 15 Oct 2017 12:51:29 +0000 (14:51 +0200)]
Merge branch 'refs/heads/t/lopsub'
Conversion to lopsub and a few other improvements on top of it.
* refs/heads/t/lopsub:
INSTALL: Explain how to use CPPFLAGS and LDFLAGS.
build: Introduce DSS_CPPFLAGS.
build: Fix cc command which creates dependencies,
build: Combine CFLAGS and DEBUG_CFLAGS.
Implement --checksum.
run: Improve error diagnostics for chdir(2) failure.
run: Improve error message if dss is already running.
run: Fix exit status in case another dss process is running.
build: Add target install and install-strip.
Convert dss to lopsub.
Remove --no-resume.
This was not a good idea because ftok(3) hashes, among other
information, the inode number of the file, and this number changes
every time the configuration file is edited.
The revert conflicted slightly to the commit which renamed
get_key_or_die() to get_key() and changed the type of the return
value to key_t, but the conflict was easy to resolve.
Andre Noll [Sun, 16 Apr 2017 10:48:58 +0000 (12:48 +0200)]
ipc: Improve error diagnostics for kill.
If dss is not running, the kill command prints "No such file or
directory" because the call to semget(2) fails with ENOENT. This
message is a bit misleading, so let's return -E_NOT_RUNNING in this
case instead.
Andre Noll [Sun, 30 Apr 2017 00:33:57 +0000 (02:33 +0200)]
Replace dss.dia by a shell script.
The dia command line tool misaligns the text on the dss logo, and the
dia application started to segfault on my home box after a library
upgrade.
This patch replaces the dia source file by the mklogo bash script
which runs the convert utility of ImageMagick to write the dss logo
in png format to stdout.
Andre Noll [Thu, 13 Jul 2017 17:43:04 +0000 (19:43 +0200)]
build: Introduce DSS_CPPFLAGS.
As with CFLAGS, it is good practice to leave CPPFLAGS unset in the
Makefile and append it to the cc command after our own flags, to give
the user a chance to override our settings.
This patch initializes DSS_CPPFLAGS with the VERSION_STRING define
which was part of the receipe and adds -Wunused macros, which is a
preprocessor flag rather than a compiler flag.
DSS_CPPFLAGS and CPPFLAGS are added to the two relevant commands,
in addition to the existing DSS_CFLAGS and CFLAGS.
Andre Noll [Thu, 13 Jul 2017 17:33:52 +0000 (19:33 +0200)]
build: Fix cc command which creates dependencies,
The command to create Makefile.deps was hardcoded as gcc in Makefile.
This patch changes the command to $(CC) and adds the usual set of
flags which we use for compiling.
Andre Noll [Thu, 13 Jul 2017 17:21:11 +0000 (19:21 +0200)]
build: Combine CFLAGS and DEBUG_CFLAGS.
We needed two sets of flags for gengetopt because the C code generated
by gengetopt would not compile cleanly with our rather strict set of
flags. With lopsub this is no longer necessary.
Moreover, it is considered good practice to not set CFLAGS at all but
to append the contents of this variable to the compile command. This
way the user may set the variable to override some of the options.
This commit gets rid of CFLAGS in favor of DSS_CFLAGS, which is just
the union of the CFLAGS and the DEBUG_CFLAGS variables we had before.
Andre Noll [Sun, 16 Apr 2017 10:01:42 +0000 (12:01 +0200)]
run: Improve error diagnostics for chdir(2) failure.
In run mode, if the destination directory does not exist, dss prints
"No such file or directory" and exits, without telling the user (a)
it was a failed chdir(2) call that caused the error, and (b) the name
of the directory. This patch adds an error message containing this
information.
Since there is only one caller of dss_chdir(), let's get rid
of this public function in file.c and call chdir() directly from
change_to_dest_dir() of dss.c.
Andre Noll [Fri, 17 Jun 2016 07:29:12 +0000 (09:29 +0200)]
run: Improve error message if dss is already running.
The current error message, "child terminated unexpectedly", is not
very comprehensive.
The most likely reason for the child to terminate is that it could not
obtain the semaphore lock because another dss process is running. This
commit adds a test to com_run() that check this condition in the
parent before the child process is born. This way, if another process
is holding the lock, we can fail with a nice error message that also
includes the pid of the process that holds the lock.
Andre Noll [Thu, 16 Jun 2016 21:21:08 +0000 (23:21 +0200)]
run: Fix exit status in case another dss process is running.
In daemon mode, we must acquire the semaphore lock in the child process
because the child does not inherit semaphore adjustments. Currently
the parent exits successfully after the fork, so the command appears
to succeed even if the child dies immediately because it was unable
to acquire the lock because another dss process is holding the lock.
This commit introduces a mechanism which enables the parent to tell
whether the child completed its setup successfully. We create a
pipe prior to calling fork(2), and let the child write to one end
of the pipe after setup is complete and just before it enters the
main select loop. The parent reads from the other end of the pipe
and exits once the read(2) call returns. If the child dies early,
read(2) returns zero, indicating failure.
Andre Noll [Sun, 16 Apr 2017 11:56:09 +0000 (13:56 +0200)]
build: Add target install and install-strip.
It has always been a bit clumsy to copy the executable and the
manual page to their proper locations by hand, so this commit adds
the two standard targets "install" and "install-strip" which install
both files.
The installation prefix defaults to /usr/local and can be set with
PREFIX. Moreover, there is DESTDIR which may be given to prepend
another directory (useful for for "staged installs", where the
installed files are not placed directly into their expected location
but are instead copied into a temporary location).
Andre Noll [Fri, 6 May 2016 14:18:24 +0000 (16:18 +0200)]
Convert dss to lopsub.
This commit ditches gengetopt for the command line and config file
parsers in favor of the lopsub library. Hence from now on, lopsub
must be installed in order to compile dss while gengetopt is no
longer needed.
The mutually exclusive gengetopt group options --create, --prune, --ls,
--run, --kill and --reload are replaced by lopsub subcommands. However,
the --reload and --kill options have been combined to the new "kill"
subcommand which allows to send arbitrary signals to a running dss
process.
Due to the conversion, the syntax of the dss command changes
slightly. For example,
dss --run
becomes
dss run
while
dss -Rdc foo
needs to be spelled as
dss -c foo -- run -d
so that -d is regarded as an option to the "run" subcommand rather
than an option to dss.
With lopsub each subcommand has its own command line and config file
parser. Options to subcommands can be added to the configuration file
like this:
[run]
daemon
logfile=/var/log/dss.log
As for the implementation, the bulk of the changes is the conversion
of dss.ggo to the new dss.suite. The necessary adjustments to the
code are relatively simple. In particular, only dss.c needs to be
changed while all other .c files don't require any modifications.
The examples in INSTALL are adjusted to the new syntax. The commit also
drops support for Mac OS and Solaris, since lopsub is not supported
on these platforms yet.
Andre Noll [Fri, 17 Feb 2017 14:40:58 +0000 (15:40 +0100)]
ipc: Prefer key_t over int for System V IPC keys.
get_key() calls ftok(3), which returns a key_t value. key_t is also
the type which semget(2), the only function which receives the key via
mutex_get(), expects. It's stupid to convert the key_t from ftok(3)
into an int, only to convert it back to key_t later.
This patch changes ipc.c to use key_t everywhere. However, in
mutex_get() we print a log message containing the value of the key,
so the format string must be adjusted accordingly. Unfortunately,
on Linux, key_t is the same as int while on FreeBSD and NetBSD it is
defined as long. To avoid a warning from the compiler we use "%lx"
in the format string and cast the value to long.
Andre Noll [Fri, 17 Feb 2017 14:29:52 +0000 (15:29 +0100)]
ipc.c: Use ftok() instead of SuperFastHash.
ftok(3) uses the identity of the named file to generate a key_t type
System V IPC key, which is easier than computing the key by hashing
the (resolved) pathname of the config file. This change allows to
get rid of the realpath() and the super_fast_hash() implementation.
If ftok(3) fails, presumably because the underlying call to stat(2)
fails, we now simply return a phony identifier, similar to what we did
before in this case. This eliminates the only possible failure path
in get_key_or_die(), so this function is renamed to get_key().
Andre Noll [Mon, 9 May 2016 09:03:45 +0000 (11:03 +0200)]
Remove --no-resume.
There is no real reason for this option. Resuming a previously
cancelled snapshot is generally a very good idea, so the option is
kind of pointless. Remove it.
Andre Noll [Fri, 17 Jun 2016 07:18:40 +0000 (09:18 +0200)]
ipc: Simplify mutex_try_lock().
There is no need to actually obtain the lock. A single semaphore
operation will do just fine. With sem_op equal to zero and IPC_NOWAIT
the semop() call returns immediately, and the return value tells
whether the semaphore value was zero.
Rename the (static) function to mutex_is_locked() to indicate that
it performs only read-only operations on the semaphore set.
Andre Noll [Thu, 16 Jun 2016 21:06:29 +0000 (23:06 +0200)]
ipc: Make pid pointer optional.
This changes get_dss_pid() to handle the case where the caller passed
a NULL pid pointer. Conversely, if pid is not NULL, we now make sure
to initialize the given address in all cases.
The single caller currently never passes NULL, so this change is just
defensive programming, protecting against future users. Be liberal
in what you accept, be strict in what you return..