qibuild.sh – Common filesystem operations

Common filesystem operations

class qibuild.sh.TempDir(name='tmp')

This is a nice wrapper around tempfile module.

Usage:

with TempDir("foo-bar") as temp_dir:
    subdir = os.path.join(temp_dir, "subdir")
    do_foo(subdir)

This piece of code makes sure that:

  • a temporary directory named temp_dir has been created (guaranteed to exist, be empty, and writeable)
  • the directory will be removed when the scope of temp_dir has ended unless an exception has occurred and DEBUG environment variable is set.
qibuild.sh.change_cwd(*args, **kwds)

Change the current working dir

qibuild.sh.configure_file(in_path, out_path, copy_only=False, *args, **kwargs)

Configure a file. :param in_path: input file :parm out_path: output file

The out_path needs not to exist, missing leading directories will be created if necessary.

If copy_only is True, the contents will be copied “as is”.

If not, we will use the args and kwargs parameter as in:

in_content.format(*args, **kwargs)
qibuild.sh.install(src, dest, filter_fun=None, quiet=False)

Install a directory to a destination.

If filter_fun is not None, then the file will only be installed if filter_fun(relative/path/to/file) returns True.

Few notes: rewriting cp or install is a hard problem. This version will happily erase whatever is inside dest, (even it the dest is readonly, dest will be erased before being written) and it won’t complain if dest does not exists (missing directories will simply be created)

This function will preserve relative symlinks between directories, used for instance in Mac frameworks:

|__ Versions
    |__ Current  -> 4.0
    |__ 4        -> 4.0
    |__ 4.0
qibuild.sh.is_executable_binary(file_path)

Returns true if the file: * is executable * is a binary (i.e not a script)

qibuild.sh.is_runtime(filename)

Filter function to only install runtime components of packages

qibuild.sh.ln(src, dst, symlink=True)

ln (do not fail if file exists)

qibuild.sh.ls_r(directory)

Returns a sorted list of all the files present in a diretory, relative to this directory.

For instance, with:

foo
|__ eggs
|    |__ c
|    |__ d
|__ empty
|__ spam
    |__a
    |__b

ls_r(foo) returns: [“eggs/c”, “eggs/d”, “empty/”, “spam/a”, “spam/b”]

qibuild.sh.mkdir(dest_dir, recursive=False)

Recursive mkdir (do not fail if file exists)

qibuild.sh.mv(src, dest)

Move a file into a directory, but do not crash if dest/src exists

qibuild.sh.rm(name)

This one can take a file or a directory. Contrary to shutil.remove or os.remove, it:

  • won’t fail if the directory does not exists
  • won’t fail if the directory contains read-only files
  • won’t fail if the file does not exists

Please avoid using shutil.rmtree ...

qibuild.sh.rmtree(path)

shutil.rmtree() on steroids.

Recursively removes a directory, even if it’s marked read-only.

shutil.rmtree() doesn’t work on Windows if any of the files or directories are read-only, which svn repositories and some .svn files are. We need to be able to force the files to be writable (i.e., deletable) as we traverse the tree.

Even with all this, Windows still sometimes fails to delete a file, citing a permission error (maybe something to do with antivirus scans or disk indexing). The best suggestion any of the user forums had was to wait a bit and try again, so we do that too. It’s hand-waving, but sometimes it works. :/

On POSIX systems, things are a little bit simpler. The modes of the files to be deleted doesn’t matter, only the modes of the directories containing them are significant. As the directory tree is traversed, each directory has its mode set appropriately before descending into it. This should result in the entire tree being removed, with the possible exception of path itself, because nothing attempts to change the mode of its parent. Doing so would be hazardous, as it’s not a directory slated for removal. In the ordinary case, this is not a problem: for our purposes, the user will never lack write permission on path‘s parent.

qibuild.sh.run(program, args)

exec a process.

  • linux: this will call exec and replace the current process
  • windows: this will call spawn and wait till the end

Example:

run("python.exe", "toto.py")
qibuild.sh.safe_copy(src, dest)

Copy a source to a destination but do not overwrite dest if it is more recent than src

Create any missing directories when necessary

If dest is a directory, src will be copied inside dest.

qibuild.sh.to_dos_path(path)

Return a DOS path from a “windows with /” path. Useful because people sometimes use forward slash in environment variable, for instance

qibuild.sh.to_native_path(path)

Return an absolute, native path from a path, (and all lower-case on case-insensitive filesystems)

qibuild.sh.to_posix_path(path)

Returns a POSIX path from a DOS path

Useful because cmake needs POSIX paths.

Guidelines:
  • always use os.path insternally
  • convert to POSIX path at the very last moment
qibuild.sh.which(program)

find program in the environment PATH :return: path to program if found, None otherwise