diff --git a/CHANGES b/CHANGES index 232a348ca..fb34dc9d6 100644 --- a/CHANGES +++ b/CHANGES @@ -17,6 +17,10 @@ This release contains contributions from (alphabetically by first name): requirements checks in the welcome module (RAM, disk space, ..). The checks have been made asynchronous, so that responsiveness during requirements-checking is improved and the user has better feedback. + * Support for building an AppImage of Calamares has been added to the + `ci/` directory. There are use-cases where a containerized build and + configuration make sense rather than having Calamares installed in the + host system. (Thanks to the AppImage team, Alexis) ## Modules ## diff --git a/ci/AppImage.md b/ci/AppImage.md new file mode 100644 index 000000000..7fa51a8bc --- /dev/null +++ b/ci/AppImage.md @@ -0,0 +1,45 @@ +# AppImage building for Calamares + +> It is possible to build Calamares as an AppImage (perhaps other +> containerized formats as well). This might make sense for +> OEM phase-1 deployments in environments where Calamares is +> not using the native toolkit. + +## AppImage tools + +You will need + - [`linuxdeploy-x86_64.AppImage`](https://github.com/linuxdeploy/linuxdeploy/releases) + - [`linuxdeploy-plugin-qt-x86_64.AppImage`](https://github.com/linuxdeploy/linuxdeploy-plugin-qt/releases) + - [`linuxdeploy-plugin-conda.sh`](https://github.com/linuxdeploy/linuxdeploy-plugin-conda) + +These tools should run -- they are bundled as AppImages after all -- on +any modern Linux system. The [AppImage packaging documentation](https://docs.appimage.org/packaging-guide/) +explains how the whole tooling works. + +If the tools are not present, the build script (see below) will download them, +but you should save them for later. + +## AppImage build + +From the **source** directory, run `ci/AppImage.sh`: + - Use `--tools-dir` to copy the tools from a local cache rather than + downloading them again. + - Run it with `--cmake-args` for special CMake handling. + - Use `--skip-build` to avoid rebuilding Calamares all the time. + - Use `--config-dir` to copy in Calamares configuration files (e.g. + *settings.conf* and the module configuration files) from a given + directory. + +The build process will: + - copy (or download) the AppImage tools into a fresh build directory + - configure and build Calamares with suitable settings + - modifies the standard `.desktop` file to be AppImage-compatible + - builds the image with the AppImage tools + +## AppImage caveats + +The resulting AppImage, `Calamares-x86_64.AppImage`, can be run as if it is +a regular Calamares executable. For internal reasons it always passes the +`-X` flag; any other command-line flags are passed in unchanged. Internally, +`XDG_*_DIRS` are used to get Calamares to find the resources inside the AppImage +rather than in the host system. diff --git a/ci/AppImage.sh b/ci/AppImage.sh new file mode 100644 index 000000000..348784755 --- /dev/null +++ b/ci/AppImage.sh @@ -0,0 +1,263 @@ +#! /bin/sh +# +# SPDX-License-Identifier: BSD-2-Clause +# +# Copyright 2019 Adriaan de Groot +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions +# are met: +# +# 1. Redistributions of source code must retain the above copyright +# notice, this list of conditions and the following disclaimer. +# 2. Redistributions in binary form must reproduce the above copyright +# notice, this list of conditions and the following disclaimer in the +# documentation and/or other materials provided with the distribution. +# +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" +# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE +# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR +# CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF +# SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS +# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN +# CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) +# ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE +# POSSIBILITY OF SUCH DAMAGE. +# +### END LICENSES + +### USAGE +# +# Shell script to help build an AppImage for Calamares. +# +# Usage: +# AppImage.sh [-T|--tools-dir ] +# [-C|--cmake-args ] +# [-c|--config-dir ] +# [-s|--skip-build] +# [-p|--with-python] +# +# Multiple --cmake-args arguments will be collected together and passed to +# CMake before building the application. +# +# Use --tools-dir to indicate where the linuxdeploy tools are located. +# +# Use --config to copy a config-directory (with settings.conf and others) +# into the resulting image, +# +# Option --skip-build assumes that there is an already-built Calamares +# available in the AppImage build directory; use this when you are, e.g. +# re-packaging the image with different configuration. Option --with-python +# adds the Conda Python packaging ecosystem to the AppImage, which will make +# it **more** portable by disconnecting from the system Python libraries. +# +# The build process for AppImage proceeds in a directory build-AppImage +# that is created in the current directory. +# +# TODO: Conda / Python support doesn't work yet. +# +### END USAGE + +TOOLS_DIR="." +CMAKE_ARGS="" +DO_REBUILD="true" +DO_CONDA="false" +CONFIG_DIR="" +while test "$#" -gt 0 +do + case "x$1" in + x--help|x-h) + sed -e '1,/USAGE/d' -e '/END.USAGE/,$d' < "$0" + return 0 + ;; + x--tools-dir|x-T) + TOOLS_DIR="$2" + shift + ;; + x--cmake-args|x-C) + CMAKE_ARGS="$CMAKE_ARGS $2" + shift + ;; + x--config-dir|x-c) + CONFIG_DIR="$2" + shift + ;; + x--skip-build|x-s) + DO_REBUILD="false" + ;; + x--with-python|x-p) + DO_CONDA="true" + ;; + *) + echo "! Unknown argument '$1'." + exit 1 + ;; + esac + test "$#" -gt 0 || { echo "! Missing arguments."; exit 1; } + shift +done + +### Check where we're running +# +BIN_DIR=$( cd $( dirname "$0" ) && pwd -P ) +test -d "$BIN_DIR" || { echo "! Could not find BIN_DIR"; exit 1; } +test -f "$BIN_DIR/AppImage.sh" || { echo "! $BIN_DIR does not have AppImage.sh"; exit 1; } + +SRC_DIR=$( cd "$BIN_DIR/.." && pwd -P ) +test -d "$SRC_DIR" || { echo "! Could not find SRC_DIR"; exit 1; } +test -d "$SRC_DIR/ci" || { echo "! $SRC_DIR isn't a top-level Calamares checkout"; exit 1; } +test -f "$SRC_DIR/CMakeLists.txt" || { echo "! SRC_DIR is missing CMakeLists.txt"; exit 1; } + +### Check pre-requisites +# +BUILD_DIR=build-AppImage +test -d "$BUILD_DIR" || mkdir -p "$BUILD_DIR" +test -d "$BUILD_DIR" || { echo "! Could not create $BUILD_DIR"; exit 1; } + +TOOLS_LIST="linuxdeploy-x86_64.AppImage linuxdeploy-plugin-qt-x86_64.AppImage" +$DO_CONDA && TOOLS_LIST="$TOOLS_LIST linuxdeploy-plugin-conda.sh" + +for tool in $TOOLS_LIST +do + if test -x "$BUILD_DIR/$tool" ; then + # This tool is ok + : + else + if test -f "$TOOLS_DIR/$tool" ; then + cp "$TOOLS_DIR/$tool" "$BUILD_DIR/$tool" || exit 1 + else + fetch=$( grep "^# URL .*$tool\$" "$0" | sed 's/# URL *//' ) + curl -L -o "$BUILD_DIR/$tool" "$fetch" + fi + chmod +x "$BUILD_DIR/$tool" + test -x "$BUILD_DIR/$tool" || { echo "! Missing tool $tool in tools-dir $TOOLS_DIR"; exit 1; } + fi +done + +if test -z "$CONFIG_DIR" ; then + echo "# Using basic settings.conf" +else + test -f "$CONFIG_DIR/settings.conf" || { echo "! No settings.conf in $CONFIG_DIR"; exit 1; } +fi + +### Clean up build-directory +# +rm -rf "$BUILD_DIR/AppDir" +if $DO_REBUILD ; then + rm -rf "$BUILD_DIR/build" + mkdir "$BUILD_DIR/build" || { echo "! Could not create $BUILD_DIR/build for the cmake-build."; exit 1; } +else + test -d "$BUILD_DIR/build" || { echo "! No build found in $BUILD_DIR, but --skip-build is given."; exit 1; } + test -x "$BUILD_DIR/build/calamares" || { echo "! No complete build found in $BUILD_DIR/build ."; exit 1; } +fi +mkdir "$BUILD_DIR/AppDir" || { echo "! Could not create $BUILD_DIR/AppDir for the AppImage install."; exit 1; } +LOG_FILE="$BUILD_DIR/AppImage.log" +rm -f "$LOG_FILE" +echo "# Calamares build started" `date` > "$LOG_FILE" + +### Python Support +# +# +if $DO_CONDA ; then + export CONDA_CHANNELS="conda-forge;anaconda" + export CONDA_PACKAGES="gettext;py-boost" + + ( + cd "$BUILD_DIR" && + ./linuxdeploy-x86_64.AppImage --appdir=AppDir/ --plugin=conda + ) + + . "$BUILD_DIR/AppDir/usr/conda/bin/activate" +fi + +### Build Calamares +# +if $DO_REBUILD ; then + echo "# Running cmake ..." + ( + cd "$BUILD_DIR/build" && + cmake "$SRC_DIR" -DCMAKE_INSTALL_PREFIX=/usr -DCMAKE_INSTALL_LIBDIR=lib $CMAKE_ARGS + ) >> "$LOG_FILE" 2>&1 || { tail -10 "$LOG_FILE" ; echo "! Could not run CMake"; exit 1; } + echo "# Running make ..." + ( + cd "$BUILD_DIR/build" && + make -j4 + ) >> "$LOG_FILE" 2>&1 || { tail -10 "$LOG_FILE" ; echo "! Could not run make"; exit 1; } +fi +echo "# Running make install ..." +( + cd "$BUILD_DIR/build" && + make install DESTDIR=../AppDir +) >> "$LOG_FILE" 2>&1 || { tail -10 "$LOG_FILE" ; echo "! Could not run make install"; exit 1; } + +### Modify installation +# +IMAGE_DIR="$BUILD_DIR/AppDir" + +# Munge the desktop file to not use absolute paths or pkexec +sed -i \ + -e 's+^Exec=.*+Exec=calamares+' \ + -e 's+^Name=.*+Name=Calamares+' \ + "$IMAGE_DIR"/usr/share/applications/calamares.desktop + +# Replace the executable with a shell-proxy +test -x "$IMAGE_DIR/usr/bin/calamares" || { echo "! Does not seem to have installed calamares"; exit 1; } +mv "$IMAGE_DIR/usr/bin/calamares" "$IMAGE_DIR/usr/bin/calamares.bin" +cat > "$IMAGE_DIR/usr/bin/calamares" <<"EOF" +#! /bin/sh +# +# Calamares proxy-script +export XDG_DATA_DIRS="$APPDIR/usr/share/calamares:" +export XDG_CONFIG_DIRS="$APPDIR/etc/calamares:$D/usr/share:" +export PYTHONPATH=$APPDIR/usr/lib: +cd "$APPDIR" +exec "$APPDIR"/usr/bin/calamares.bin -X "$@" +EOF +chmod 755 "$IMAGE_DIR/usr/bin/calamares" +test -x "$IMAGE_DIR/usr/bin/calamares" || { echo "! Does not seem to have proxy for calamares"; exit 1; } + +### Install additional files +# +PLUGIN_DIR=$( qmake -query QT_INSTALL_PLUGINS ) +for plugin in \ + libpmsfdiskbackendplugin.so \ + libpmdummybackendplugin.so +do + cp "$PLUGIN_DIR/$plugin" "$IMAGE_DIR/usr/lib" || { echo "! Could not copy plugin $plugin"; exit 1; } +done + +# Install configuration files +ETC_DIR="$IMAGE_DIR"/etc/calamares +mkdir -p "$ETC_DIR" +test -d "$ETC_DIR" || { echo "! Could not create /etc/calamares in image."; exit 1; } + +if test -z "$CONFIG_DIR" ; then + echo "# Using basic settings.conf" + cp "$SRC_DIR/settings.conf" "$ETC_DIR" +else + test -f "$CONFIG_DIR/settings.conf" || { echo "! No settings.conf in $CONFIG_DIR"; exit 1; } + mkdir -p "$ETC_DIR/modules" + cp "$CONFIG_DIR/settings.conf" "$ETC_DIR" + test -d "$CONFIG_DIR/modules" && cp -r "$CONFIG_DIR/modules" "$ETC_DIR" + test -d "$CONFIG_DIR/branding" && cp -r "$CONFIG_DIR/branding" "$IMAGE_DIR/usr/share/calamares" +fi + +### Build the AppImage +# +# +echo "# Building AppImage" +( + export QT_SELECT=qt5 # Otherwise might pick Qt4 in image + export LD_LIBRARY_PATH=AppDir/usr/lib # RPATH isn't set in the executable + cd "$BUILD_DIR" && + ./linuxdeploy-x86_64.AppImage --appdir=AppDir/ --plugin=qt --output=appimage +) >> "$LOG_FILE" 2>&1 || { tail -10 "$LOG_FILE" ; echo "! Could not create image"; exit 1; } + +exit 0 +### Database for installation +# +# URL https://github.com/linuxdeploy/linuxdeploy/releases/download/continuous/linuxdeploy-x86_64.AppImage +# URL https://github.com/linuxdeploy/linuxdeploy-plugin-qt/releases/download/continuous/linuxdeploy-plugin-qt-x86_64.AppImage +# URL https://raw.githubusercontent.com/TheAssassin/linuxdeploy-plugin-conda/master/linuxdeploy-plugin-conda.sh diff --git a/data/config-appimage/branding/default/squid.png b/data/config-appimage/branding/default/squid.png new file mode 100644 index 000000000..dbe615c18 Binary files /dev/null and b/data/config-appimage/branding/default/squid.png differ diff --git a/data/config-appimage/modules/displaymanager.conf b/data/config-appimage/modules/displaymanager.conf new file mode 100644 index 000000000..8f8e9c704 --- /dev/null +++ b/data/config-appimage/modules/displaymanager.conf @@ -0,0 +1,28 @@ +# Configure one or more display managers (e.g. SDDM) +# with a "best effort" approach. +--- +#The DM module attempts to set up all the DMs found in this list, in that precise order. +#It also sets up autologin, if the feature is enabled in globalstorage. +#The displaymanagers list can also be set in globalstorage, and in that case it overrides anything set up here. +displaymanagers: + - slim + - sddm + - lightdm + - gdm + - mdm + - lxdm + - kdm + +#Enable the following settings to force a desktop environment in your displaymanager configuration file: +#defaultDesktopEnvironment: +# executable: "startkde" +# desktopFile: "plasma" + +#If true, try to ensure that the user, group, /var directory etc. for the +#display manager are set up correctly. This is normally done by the distribution +#packages, and best left to them. Therefore, it is disabled by default. +basicSetup: false + +#If true, setup autologin for openSUSE. This only makes sense on openSUSE +#derivatives or other systems where /etc/sysconfig/displaymanager exists. +sysconfigSetup: false diff --git a/data/config-appimage/modules/finished.conf b/data/config-appimage/modules/finished.conf new file mode 100644 index 000000000..29e5e49b4 --- /dev/null +++ b/data/config-appimage/modules/finished.conf @@ -0,0 +1,21 @@ +# Configuration for the "finished" page, which is usually shown only at +# the end of the installation (successful or not). +--- +# The finished page can hold a "restart system now" checkbox. +# If this is false, no checkbox is shown and the system is not restarted +# when Calamares exits. +restartNowEnabled: true + +# Initial state of the checkbox "restart now". Only relevant when the +# checkbox is shown by restartNowEnabled. +restartNowChecked: false + +# If the checkbox is shown, and the checkbox is checked, then when +# Calamares exits from the finished-page it will run this command. +# If not set, falls back to "shutdown -r now". +restartNowCommand: "systemctl -i reboot" + +# When the last page is (successfully) reached, send a DBus notification +# to the desktop that the installation is done. This works only if the +# user as whom Calamares is run, can reach the regular desktop session bus. +notifyOnFinished: false diff --git a/data/config-appimage/modules/keyboard.conf b/data/config-appimage/modules/keyboard.conf new file mode 100644 index 000000000..ee97c3939 --- /dev/null +++ b/data/config-appimage/modules/keyboard.conf @@ -0,0 +1,16 @@ +# NOTE: you must have ckbcomp installed and runnable +# on the live system, for keyboard layout previews. +--- +# The name of the file to write X11 keyboard settings to +# The default value is the name used by upstream systemd-localed. +# Relative paths are assumed to be relative to /etc/X11/xorg.conf.d +xOrgConfFileName: "/etc/X11/xorg.conf.d/00-keyboard.conf" + +# The path to search for keymaps converted from X11 to kbd format +# Leave this empty if the setting does not make sense on your distribution. +convertedKeymapPath: "/lib/kbd/keymaps/xkb" + +# Write keymap configuration to /etc/default/keyboard, usually +# found on Debian-related systems. +# Defaults to true if nothing is set. +#writeEtcDefaultKeyboard: true diff --git a/data/config-appimage/modules/locale.conf b/data/config-appimage/modules/locale.conf new file mode 100644 index 000000000..8ae016279 --- /dev/null +++ b/data/config-appimage/modules/locale.conf @@ -0,0 +1,31 @@ +--- +# This settings are used to set your default system time zone. +# Time zones are usually located under /usr/share/zoneinfo and +# provided by the 'tzdata' package of your Distribution. +# +# Distributions using systemd can list available +# time zones by using the timedatectl command. +# timedatectl list-timezones +# +# The starting timezone (e.g. the pin-on-the-map) when entering +# the locale page can be set through keys *region* and *zone*. +# If either is not set, defaults to America/New_York. +# +region: "Europe" +zone: "Amsterdam" + + +# Enable only when your Distribution is using an +# custom path for locale.gen +#localeGenPath: "PATH_TO/locale.gen" + +# GeoIP based Language settings: +# +# GeoIP need an working Internet connection. +# +geoipUrl: "https://geoip.kde.org/v1/calamares" + +# GeoIP style. Leave commented out for the "legacy" interpretation. +# This setting only makes sense if geoipUrl is set, enabliing geoIP. +geoipStyle: "json" + diff --git a/data/config-appimage/modules/users.conf b/data/config-appimage/modules/users.conf new file mode 100644 index 000000000..bdf812878 --- /dev/null +++ b/data/config-appimage/modules/users.conf @@ -0,0 +1,59 @@ +# Configuration for the one-user-system user module. +# +# Besides these settings, the user module also places the following +# keys into the globalconfig area, based on user input in the view step. +# +# - hostname +# - username +# - password (obscured) +# - autologinUser (if enabled, set to username) +# +# These globalconfig keys are set when the jobs for this module +# are created. +--- +# Used as default groups for the created user. +# Adjust to your Distribution defaults. +defaultGroups: + - users + - lp + - video + - network + - storage + - wheel + - audio + +# Some Distributions require a 'autologin' group for the user. +# Autologin causes a user to become automatically logged in to +# the desktop environment on boot. +# Disable when your Distribution does not require such a group. +autologinGroup: autologin +# You can control the initial state for the 'autologin checkbox' in UsersViewStep here. +# Possible values are: true to enable or false to disable the checkbox by default +doAutologin: true + +# When set to a non-empty string, Calamares creates a sudoers file for the user. +# /etc/sudoers.d/10-installer +# Remember to add sudoersGroup to defaultGroups. +# +# If your Distribution already sets up a group of sudoers in its packaging, +# remove this setting (delete or comment out the line below). Otherwise, +# the setting will be duplicated in the /etc/sudoers.d/10-installer file, +# potentially confusing users. +sudoersGroup: wheel + +# Setting this to false , causes the root account to be disabled. +setRootPassword: true +# You can control the initial state for the 'root password checkbox' in UsersViewStep here. +# Possible values are: true to enable or false to disable the checkbox by default. +# When enabled the user password is used for the root account too. +# NOTE: doReusePassword requires setRootPassword to be enabled. +doReusePassword: true + +# These are optional password-requirements that a distro can enforce +# on the user. The values given in this sample file disable each check, +# as if the check was not listed at all. +passwordRequirements: + minLength: -1 # Password at least this many characters + maxLength: -1 # Password at most this many characters + +userShell: /bin/bash diff --git a/data/config-appimage/modules/welcome.conf b/data/config-appimage/modules/welcome.conf new file mode 100644 index 000000000..8dae3e957 --- /dev/null +++ b/data/config-appimage/modules/welcome.conf @@ -0,0 +1,46 @@ +# Configuration for the welcome module. The welcome page +# displays some information from the branding file. +# Which parts it displays can be configured through +# the show* variables. +# +# In addition to displaying the welcome page, this module +# can check requirements for installation. +--- +# Display settings for various buttons on the welcome page. +showSupportUrl: true +showKnownIssuesUrl: true +showReleaseNotesUrl: true + +# Requirements checking. These are general, generic, things +# that are checked. They may not match with the actual requirements +# imposed by other modules in the system. +requirements: + # Amount of available disk, in GB. Floating-point is allowed here. + # Note that this does not account for *usable* disk, so it is possible + # to pass this requirement, yet have no space to install to. + requiredStorage: 5.5 + + # Amount of available RAM, in GB. Floating-point is allowed here. + requiredRam: 1.0 + + # To check for internet connectivity, Calamares does a HTTP GET + # on this URL; on success (e.g. HTTP code 200) internet is OK. + internetCheckUrl: http://google.com + + # List conditions to check. Each listed condition will be + # probed in some way, and yields true or false according to + # the host system satisfying the condition. + # + # This sample file lists all the conditions that are known. + check: + - ram + - power + - internet + - root + - screen + # List conditions that **must** be satisfied (from the list + # of conditions, above) for installation to proceed. + # If any of these conditions are not met, the user cannot + # continue past the welcome page. + required: + - ram diff --git a/data/config-appimage/settings.conf b/data/config-appimage/settings.conf new file mode 100644 index 000000000..756710492 --- /dev/null +++ b/data/config-appimage/settings.conf @@ -0,0 +1,36 @@ +# Configuration file for Calamares +# Syntax is YAML 1.2 +--- +modules-search: [ usr/lib/calamares/modules ] + +# YAML: list of maps of string:string key-value pairs. +#instances: +#- id: owncloud +# module: webview +# config: owncloud.conf + +# Sequence section. This section describes the sequence of modules, both +# viewmodules and jobmodules, as they should appear and/or run. +sequence: +- show: + - welcome + - locale + - keyboard + - users + - summary +- exec: + - dummypython + - locale + - keyboard + - users + - displaymanager + - networkcfg +- show: + - finished + +branding: default + +prompt-install: false +# OEM mode +dont-chroot: true +disable-cancel: false diff --git a/src/calamares/CalamaresApplication.cpp b/src/calamares/CalamaresApplication.cpp index c9a171fd2..989290567 100644 --- a/src/calamares/CalamaresApplication.cpp +++ b/src/calamares/CalamaresApplication.cpp @@ -263,7 +263,12 @@ CalamaresApplication::initSettings() ::exit( EXIT_FAILURE ); } - new Calamares::Settings( settingsFile.absoluteFilePath(), isDebug(), this ); + auto* settings = new Calamares::Settings( settingsFile.absoluteFilePath(), isDebug(), this ); // Creates singleton + if ( settings->modulesSequence().count() < 1 ) + { + cError() << "FATAL: no sequence set."; + ::exit( EXIT_FAILURE ); + } } diff --git a/src/libcalamares/Settings.cpp b/src/libcalamares/Settings.cpp index 3a00399f4..9bedbbe41 100644 --- a/src/libcalamares/Settings.cpp +++ b/src/libcalamares/Settings.cpp @@ -74,6 +74,112 @@ Settings::instance() return s_instance; } +static void +interpretModulesSearch( const bool debugMode, const QStringList& rawPaths, QStringList& output ) +{ + for ( const auto& path : rawPaths ) + { + if ( path == "local" ) + { + cDebug() << "module-search local"; + + // If we're running in debug mode, we assume we might also be + // running from the build dir, so we add a maximum priority + // module search path in the build dir. + if ( debugMode ) + { + QString buildDirModules = QDir::current().absolutePath() + + QDir::separator() + "src" + + QDir::separator() + "modules"; + if ( QDir( buildDirModules ).exists() ) + output.append( buildDirModules ); + } + + // Install path is set in CalamaresAddPlugin.cmake + output.append( CalamaresUtils::systemLibDir().absolutePath() + + QDir::separator() + "calamares" + + QDir::separator() + "modules" ); + } + else + { + QDir d( path ); + if ( d.exists() && d.isReadable() ) + { + cDebug() << "module-search exists" << d.absolutePath(); + output.append( d.absolutePath() ); + } + else + cDebug() << "module-search non-existent" << path; + } + } +} + +static void +interpretInstances( const YAML::Node& node, Settings::InstanceDescriptionList& customInstances ) +{ + // Parse the custom instances section + if ( node ) + { + QVariant instancesV = CalamaresUtils::yamlToVariant( node ).toList(); + if ( instancesV.type() == QVariant::List ) + { + const auto instances = instancesV.toList(); + for ( const QVariant& instancesVListItem : instances ) + { + if ( instancesVListItem.type() != QVariant::Map ) + continue; + QVariantMap instancesVListItemMap = + instancesVListItem.toMap(); + Settings::InstanceDescription instanceMap; + for ( auto it = instancesVListItemMap.constBegin(); + it != instancesVListItemMap.constEnd(); ++it ) + { + if ( it.value().type() != QVariant::String ) + continue; + instanceMap.insert( it.key(), it.value().toString() ); + } + customInstances.append( instanceMap ); + } + } + } +} + +static void +interpretSequence( const YAML::Node& node, Settings::ModuleSequence& moduleSequence ) +{ + // Parse the modules sequence section + if ( node ) + { + QVariant sequenceV = CalamaresUtils::yamlToVariant( node ); + if ( !( sequenceV.type() == QVariant::List ) ) + throw YAML::Exception( YAML::Mark(), "sequence key does not have a list-value" ); + + const auto sequence = sequenceV.toList(); + for ( const QVariant& sequenceVListItem : sequence ) + { + if ( sequenceVListItem.type() != QVariant::Map ) + continue; + QString thisActionS = sequenceVListItem.toMap().firstKey(); + ModuleAction thisAction; + if ( thisActionS == "show" ) + thisAction = ModuleAction::Show; + else if ( thisActionS == "exec" ) + thisAction = ModuleAction::Exec; + else + continue; + + QStringList thisActionRoster = sequenceVListItem + .toMap() + .value( thisActionS ) + .toStringList(); + moduleSequence.append( qMakePair( thisAction, + thisActionRoster ) ); + } + } + else + throw YAML::Exception( YAML::Mark(), "sequence key is missing" ); +} + Settings::Settings( const QString& settingsFilePath, bool debugMode, QObject* parent ) @@ -94,92 +200,9 @@ Settings::Settings( const QString& settingsFilePath, YAML::Node config = YAML::Load( ba.constData() ); Q_ASSERT( config.IsMap() ); - QStringList rawPaths; - config[ "modules-search" ] >> rawPaths; - for ( int i = 0; i < rawPaths.length(); ++i ) - { - if ( rawPaths[ i ] == "local" ) - { - // If we're running in debug mode, we assume we might also be - // running from the build dir, so we add a maximum priority - // module search path in the build dir. - if ( debugMode ) - { - QString buildDirModules = QDir::current().absolutePath() + - QDir::separator() + "src" + - QDir::separator() + "modules"; - if ( QDir( buildDirModules ).exists() ) - m_modulesSearchPaths.append( buildDirModules ); - } - - // Install path is set in CalamaresAddPlugin.cmake - m_modulesSearchPaths.append( CalamaresUtils::systemLibDir().absolutePath() + - QDir::separator() + "calamares" + - QDir::separator() + "modules" ); - } - else - { - QDir path( rawPaths[ i ] ); - if ( path.exists() && path.isReadable() ) - m_modulesSearchPaths.append( path.absolutePath() ); - } - } - - // Parse the custom instances section - if ( config[ "instances" ] ) - { - QVariant instancesV - = CalamaresUtils::yamlToVariant( config[ "instances" ] ).toList(); - if ( instancesV.type() == QVariant::List ) - { - const auto instances = instancesV.toList(); - for ( const QVariant& instancesVListItem : instances ) - { - if ( instancesVListItem.type() != QVariant::Map ) - continue; - QVariantMap instancesVListItemMap = - instancesVListItem.toMap(); - QMap< QString, QString > instanceMap; - for ( auto it = instancesVListItemMap.constBegin(); - it != instancesVListItemMap.constEnd(); ++it ) - { - if ( it.value().type() != QVariant::String ) - continue; - instanceMap.insert( it.key(), it.value().toString() ); - } - m_customModuleInstances.append( instanceMap ); - } - } - } - - // Parse the modules sequence section - Q_ASSERT( config[ "sequence" ] ); // It better exist! - { - QVariant sequenceV - = CalamaresUtils::yamlToVariant( config[ "sequence" ] ); - Q_ASSERT( sequenceV.type() == QVariant::List ); - const auto sequence = sequenceV.toList(); - for ( const QVariant& sequenceVListItem : sequence ) - { - if ( sequenceVListItem.type() != QVariant::Map ) - continue; - QString thisActionS = sequenceVListItem.toMap().firstKey(); - ModuleAction thisAction; - if ( thisActionS == "show" ) - thisAction = ModuleAction::Show; - else if ( thisActionS == "exec" ) - thisAction = ModuleAction::Exec; - else - continue; - - QStringList thisActionRoster = sequenceVListItem - .toMap() - .value( thisActionS ) - .toStringList(); - m_modulesSequence.append( qMakePair( thisAction, - thisActionRoster ) ); - } - } + interpretModulesSearch( debugMode, CalamaresUtils::yamlToStringList( config[ "modules-search" ] ), m_modulesSearchPaths ); + interpretInstances( config[ "instances" ], m_customModuleInstances ); + interpretSequence( config[ "sequence" ], m_modulesSequence ); m_brandingComponentName = requireString( config, "branding" ); m_promptInstall = requireBool( config, "prompt-install", false ); diff --git a/src/libcalamares/utils/YamlUtils.cpp b/src/libcalamares/utils/YamlUtils.cpp index e7eb8fd46..b9b3425e6 100644 --- a/src/libcalamares/utils/YamlUtils.cpp +++ b/src/libcalamares/utils/YamlUtils.cpp @@ -109,6 +109,14 @@ yamlMapToVariant( const YAML::Node& mapNode ) return vm; } +QStringList +yamlToStringList(const YAML::Node& listNode) +{ + QStringList l; + listNode >> l; + return l; +} + void explainYamlException( const YAML::Exception& e, const QByteArray& yamlData, const char *label ) diff --git a/src/libcalamares/utils/YamlUtils.h b/src/libcalamares/utils/YamlUtils.h index 49c8d6613..0fa48e270 100644 --- a/src/libcalamares/utils/YamlUtils.h +++ b/src/libcalamares/utils/YamlUtils.h @@ -32,6 +32,7 @@ class Node; class Exception; } +/// @brief Appends all te elements of @p node to the string list @p v void operator>>( const YAML::Node& node, QStringList& v ); namespace CalamaresUtils @@ -51,6 +52,9 @@ QVariant yamlScalarToVariant( const YAML::Node& scalarNode ); QVariant yamlSequenceToVariant( const YAML::Node& sequenceNode ); QVariant yamlMapToVariant( const YAML::Node& mapNode ); +/// @brief Returns all the elements of @p listNode in a StringList +QStringList yamlToStringList( const YAML::Node& listNode ); + /// @brief Save a @p map to @p filename as YAML bool saveYaml( const QString& filename, const QVariantMap& map ); diff --git a/src/libcalamaresui/modulesystem/Module.cpp b/src/libcalamaresui/modulesystem/Module.cpp index 20e0517ea..73a026fb9 100644 --- a/src/libcalamaresui/modulesystem/Module.cpp +++ b/src/libcalamaresui/modulesystem/Module.cpp @@ -169,7 +169,8 @@ moduleConfigurationCandidates( bool assumeBuildDir, const QString& moduleName, c void Module::loadConfigurationFile( const QString& configFileName ) //throws YAML::Exception { - foreach ( const QString& path, moduleConfigurationCandidates( Settings::instance()->debugMode(), m_name, configFileName ) ) + QStringList configCandidates = moduleConfigurationCandidates( Settings::instance()->debugMode(), m_name, configFileName ); + for ( const QString& path : configCandidates ) { QFile configFile( path ); if ( configFile.exists() && configFile.open( QFile::ReadOnly | QFile::Text ) ) @@ -198,6 +199,7 @@ Module::loadConfigurationFile( const QString& configFileName ) //throws YAML::Ex return; } } + cDebug() << "No config file found in" << Logger::DebugList( configCandidates ); } diff --git a/src/libcalamaresui/modulesystem/ModuleManager.cpp b/src/libcalamaresui/modulesystem/ModuleManager.cpp index d3705729c..b8dbc4ded 100644 --- a/src/libcalamaresui/modulesystem/ModuleManager.cpp +++ b/src/libcalamaresui/modulesystem/ModuleManager.cpp @@ -325,36 +325,47 @@ ModuleManager::checkRequirements() QTimer::singleShot( 0, rq, &RequirementsChecker::run ); } +static QStringList +missingRequiredModules( const QStringList& required, const QMap< QString, QVariantMap >& available ) +{ + QStringList l; + for( const QString& depName : required ) + { + if ( !available.contains( depName ) ) + l.append( depName ); + } + + return l; +} + QStringList ModuleManager::checkDependencies() { QStringList failed; + bool somethingWasRemovedBecauseOfUnmetDependencies = false; // This goes through the map of available modules, and deletes those whose // dependencies are not met, if any. - forever + do { - bool somethingWasRemovedBecauseOfUnmetDependencies = false; + somethingWasRemovedBecauseOfUnmetDependencies = false; for ( auto it = m_availableDescriptorsByModuleName.begin(); it != m_availableDescriptorsByModuleName.end(); ++it ) { - foreach ( const QString& depName, - it->value( "requiredModules" ).toStringList() ) + QStringList unmet = missingRequiredModules( it->value( "requiredModules" ).toStringList(), m_availableDescriptorsByModuleName ); + + if ( unmet.count() > 0 ) { - if ( !m_availableDescriptorsByModuleName.contains( depName ) ) - { - QString moduleName = it->value( "name" ).toString(); - somethingWasRemovedBecauseOfUnmetDependencies = true; - m_availableDescriptorsByModuleName.erase( it ); - failed << moduleName; - cWarning() << "Module" << moduleName << "has unknown requirement" << depName; - break; - } + QString moduleName = it->value( "name" ).toString(); + somethingWasRemovedBecauseOfUnmetDependencies = true; + m_availableDescriptorsByModuleName.erase( it ); + failed << moduleName; + cWarning() << "Module" << moduleName << "has unknown requirements" << Logger::DebugList( unmet ); + break; } } - if ( !somethingWasRemovedBecauseOfUnmetDependencies ) - break; } + while( somethingWasRemovedBecauseOfUnmetDependencies ); return failed; }