qi::File

Include: #include <qicore/file.hpp>

Summary

qi::File provides a way to open a file in read-only mode on the local file system and share the content of this file with potentially remote services.

We provide algorithms to copy a remote file’s content to make a local copy of it, using the qi::copyToLocal or qi::FileOperation algorithms.

Detailed Usage Description

Opening a file in read-only mode for sharing access

Use qi::openLocalFile to open the file in sharing read-only mode:

qi::FilePtr file = qi::openLocalFile("valid/path/to/file.ext");

In this example, the file object can be passed to a service API requesting a qi::FilePtr, making the content of the file available to the service even if remote.

If the file does not exist, a std::runtime_error exception will be thrown.

File access lifetime

The lifetime of the file access is extended by the shared ownership between all the qi::FilePtr, even if the instance is remote.

The qi::File object is kept alive and the file access open by default until all qi::FilePtr objects referring owning it are destroyed. The file access is automatically closed when qi::File is destroyed.

Once you don’t need it anymore and want to explicitly stop owning the access, you can use the qi::File::reset member of qi::FilePtr. The file access will be closed if there is no other owner still using it.

void someServiceOperation(qi::FilePtr file)
{
  // Do some work  with file...

  // No need to work with the file anymore.
  file.reset(); // From now on, `file` is not usable anymore.
  // If we were the only one accessing this file,
  // access to it's content is now closed.

  //...

  file->isOpen(); // ERROR
}

Request or provide read-only access to the content of a file

A service can request read-only access to the content of a file by using qi::FilePtr as argument.

Similarly, a service can provide read-only access to a file by having a function return a qi::FilePtr.

For example:

class MY_API MyService
{
  //...
public:

  // Store a copy of the provided file, associated with the provided name.
  void storeFile(qi::FilePtr fileToStore, const std::string& name);

  // Provide access to a stored file found by name.
  qi::FilePtr getFile(const std::string& name);

  //...
};

typedef qi::Object<MyService> MyServicePtr;

In this example, the user of the service can provide access to a file this way:

void backup(MyServicePtr service, const std::string& name, const qi::Path& fileToStore)
{
  qi::FilePtr file = qi::openLocalFile(fileToStore);
  // Now we can share reading access to this file.
  service->storeFile(file, name);
  // At this point, the service have a copy of the file and can start to work with it.
}

and acquire access to a file content provided by the service this way:

void restore(MyServicePtr service, const std::string& name)
{
  // Get Access to the file.
  qi::FilePtr file = service->getFile(name);

  if(file)
  {
    // Then work with the file content.
    // ...
  }
}

Make a local copy of a provided file

We can use the qi::copyToLocal function to make a local copy of the file provided through a qi::FilePtr. The qi::copyToLocal function will transfer a copy of the content of the file to the local file system, even if the original file is on a remote file system.

For example, if we want a service function to make a full copy of a provided file, in a local temporary location, before doing any more work:

void MyService::storeFile(qi::FilePtr fileToStore, const std::string& name)
{
  // First, make a local copy in a temporary files directory:
  const qi::Path tempFilePath = generateTemporaryFilePath();
  qi::copyToLocal(fileToStore, tempFilePath); // This call wait for the end of the copy.

  // We now have a local copy of the remote file,
  // we don't need the remote access anymore.
  fileToStore.reset();

  // Now we can work with the local copy of the file:
  storeFileInDatabase(name, tempFilePath);
}

In the same way, we can provide access to the data of the file by returning a qi::FilePtr:

FilePtr MyService::getFile(const std::string& name)
{
  const qi::Path fileLocation = findFileLocation(name);

  if (fileLocation.exists() && fileLocation.isfile())
  {
    // Now we can open it and provide it to the user for reading.
    return qi::openLocalFile(fileLocation);
  }

  // We didn't find the file!
  return FilePtr();
}

Then the user can retrieve a copy of the file and work with it:

void restore(MyServicePtr service, const std::string& name, const qi::Path& fileLocation)
{
  // Get access to the file.
  qi::FilePtr file = service->getFile(name);

  if (file) // The file was found.
  {
    // We want to make a local copy before working on the file.
    qi::copyToLocal(file, fileLocation);

    // We don't need the remote access anymore.
    file.reset();

    // Now work on the file located at `fileLocation`.
    workOnFile(fileLocation);
  }
}

Observe the progression of a file copy, transfer or other long operations

Potentially long file operations like qi::copyToLocal returns a qi::FutureSync. This imply that the call will wait for the end of the operation if the qi::FutureSync is not used by the user or not put into a qi::Future for later usage.

To wait for the end of the operation, we just wait for the future to be set:

void printFinished()
{
  qiLogInfo() << "File Transfer Finished! ";
}

void fetchFile(qi::FilePtr file)
{
    const qi::Path temporaryFilePath = makeTemporaryFilePath();

    // We copy the file on a temporary work location on the local filesystem,
    // but we don't wait for the transfer to end.
    qi::Future<void> transfer = qi::copyToLocal(file, temporaryFilePath);

    // We want to log the end of the transfer:
    transfer.connect(&printFinished);

    // Do some additional work while the file is transfered.
    someAdditionalWork();

    // Now we wait until the end of the operation if it's not finished yet.
    transfer.wait(); // Don't wait for futures in real code, you should .connect() instead.
    file.reset();

    workOnFile(temporaryFilePath);
}

In this example we want to be notified about the end of the operation, so we can connect a callback to the future. However, using only a future, we cannot get progress information about the transfer.

To get progress information and be able to connect callbacks to the different steps of the operation before the operation starts, we can use the qi::FileCopyToLocal type instead of qi::copyToLocal function:

void printStatus(qi::FileOperation::Status status)
{
  switch(status)
  {
    case Status_Finished:
      qiLogInfo() << "File Transfer Finished! ";
      break;

    case Status_Failed:
      qiLogInfo() << "File Transfer Failed! ";
      break;

     case Status_Canceled:
      qiLogInfo() << "File Transfer Canceled! ";
      break;
  }
  // We ignore the other status.
}

void printProgress(double progress)
{
  qiLogInfo() << "#### File Transfer Progress = " << (progress * 100.0) << "%";
}

void fetchFile(qi::FilePtr file)
{
    const qi::Path temporaryFilePath = makeTemporaryFilePath();

    // Instead of launching the copy immediately, we prepare it by setting up
    // the object which represent that operation.
    qi::FileCopyToLocal fileCopy {file, temporaryFilePath};

    // We extract the progress notifier associated with the operation.
    qi::ProgressNotifierPtr notifier = fileCopy->notifier();

    // We want to log the state changes of the transfer.
    notifier->status.connect(&printStatus);

    // We also to log the progress of the transfer.
    notifier->progress.connect(&printProgress);

    // We are ready to launch the copy now.
    Future<void> ft = fileCopy.start();

    // Do some additional work while the file is transfered.
    someAdditionalWork();

    // Now we wait until the end of the operation if it's not finished yet.
    ft.wait(); // Don't wait for futures in real code, you should .connect() instead.

    // We don't need the file access anymore.
    file.reset();

    workOnFile(temporaryFilePath);
}

The qi::FileCopyToLocal type is an implementation of qi::FileOperation which represents the copy/transfer operation but does not start. As this example demonstrates, the user can then plug callbacks to signals and properties provided by qi::ProgressNotifier which is provided by the qi::FileOperation::notifier() member function. The user can then start the copy operation by calling the qi::FileOperation::start() member function.

Similarly, the side of the code which opened the file and provided access to it can also follow the progression of the on-going operations on the file by using the qi::File::operationProgress() member function. This function provides access to a qi::ProgressNotifier which is updated by the current file operation on the file.

void printStatus(qi::FileOperation::Status status); // as before
void printProgress(double progress); // as before

void storeFile(MyServicePtr service, const std::string& name, const qi::Path& fileToStore)
{
  qi::FilePtr file = qi::openLocalFile(fileToStore);

  // Before sending the access to the file, connect progress logging callbacks.
  qi::ProgressNotifierPtr copyProgress = file->operationProgress();
  copyProgress->status.connect(&printStatus);
  copyProgress->progress.connect(&printProgress);

  // Now we can share reading access to this file.
  service->storeFile(file, name);

  // At this point, the service have a copy of the file and can start to work with it.
}

Complete example : Alice’s Image Store Service

The following code samples are extracted from the complete example project located in the worktree at sdk/libqicore/libqicore/example/file/

In this example, Alice design a naoqi service which provide storage for images files. Then Bob writes an application which access Alice’s image storage service and work with it.

ImageStore

Alice’s ImageStore service is simple and straightforward:

#ifndef ALICE_SERVICE_IMAGESTORE_HPP
#define ALICE_SERVICE_IMAGESTORE_HPP
#pragma once

#include <qicore/file.hpp>

#include "api.hpp"

namespace alice
{
class ALICE_SERVICE_API ImageStore
{
public:
  virtual ~ImageStore() = default;

  // Store a copy of the image file and associate it with the provided name.
  virtual void storeImage(qi::FilePtr imageFile, std::string name) = 0;

  // Provide access to an image file associated with the provided name.
  virtual qi::FilePtr getImage(std::string name) = 0;

protected:
  ImageStore() = default;
};

using ImageStorePtr = qi::Object<ImageStore>;

ALICE_SERVICE_API ImageStorePtr getImageStore();
}
#endif

This is very similar to the example code in previous sections. In particular:

  // Store a copy of the image file and associate it with the provided name.
  virtual void storeImage(qi::FilePtr imageFile, std::string name) = 0;

  // Provide access to an image file associated with the provided name.
  virtual qi::FilePtr getImage(std::string name) = 0;

Note the qi::FilePtr usage which allow the user to provide access to a file, or to acquire a stored file access.

To implement the storeImage function, Alice first acquire and copy the user file in a temporary location, then do some work to register it in the store’s database.

    // but for simplicity we will do it synchronously.

    // First, make a local copy in a temporary files directory:
    const auto tempFilePath = generateTemporaryFilePath();

    // This call will block until the end because it returns a FutureSync.
    qi::copyToLocal(imageFile, tempFilePath);

    // We now have a local copy of the remote file,
    // so we don't need the remote access anymore.
    imageFile.reset();

    // Now we can work with the local file.
    storeFileInDatabase(name, tempFilePath);
  }

  qi::FilePtr getImage(std::string name) override
  {

This is of course a simpler implementation, similar to the getImage function implementation:

    // Now we can open it and provide it to the user for reading.
    return qi::openLocalFile(fileLocation);
  }

private:
  using FileRegistry = std::map<std::string, qi::Path>;

ImageStore User Code

Meanwhile, Bob wrote his application to acquire Alice’s ImageStore instance, than do some basic work with it.

namespace bob
{
void printTranferProgress(double progress)
{
  qiLogInfo() << ">>>> File Transfer Progress = " << (progress * 100.0) << "%";
}

void workOnImageFile(const qi::Path& imageFilePath)
{
  qiLogInfo() << "Working on image file at " << imageFilePath << " ...";
  // we fake working on it...
  boost::this_thread::sleep_for(boost::chrono::milliseconds(300));
  qiLogInfo() << "Working on image file at " << imageFilePath << " - DONE";
}

void storeImage(alice::ImageStorePtr imageStore, const std::string& name, const qi::Path& imageFilePath)
{
  qiLogInfo() << "Storing image file at " << imageFilePath << " into the ImageStore...";

  // First open the file with read-only shareable access.
  qi::FilePtr file = qi::openLocalFile(imageFilePath);

  // Now we can share reading access to this file.
  imageStore->storeImage(file, name);

  qiLogInfo() << "Storing image file at " << imageFilePath << " into the ImageStore - DONE";
}

void processImage(alice::ImageStorePtr imageStore, const std::string& imageFile, const qi::Path& imageFilePath)
{
  // We acquire read-only access to the file and retrieve it locally.
  qi::FilePtr file = imageStore->getImage(imageFile);
  qi::copyToLocal(file, imageFilePath);

  // We don't need the remote access anymore.
  file.reset();

  // Now work on the file located at `fileLocation`.
  workOnImageFile(imageFilePath);
}

void processImageWithProgress(alice::ImageStorePtr imageStore,
                              const std::string& imageFile,
                              const std::string& imageFilePath)
{
  // We acquire read-only access to the file.
  qi::FilePtr file = imageStore->getImage(imageFile);

  // We prepare the operation without launching it yet:
  qi::FileCopyToLocal fileOp{file, imageFilePath};

  // We want to see the progress so we plug a logging function.
  fileOp.notifier()->progress.connect(&printTranferProgress);

  // Launch the copy and wait for it to end before continuing.
  fileOp.start().wait(); // Don't wait for futures in real code, you should .connect() instead.

  // We don't need the remote access anymore.
  file.reset();

  // Now work on the file located at `fileLocation`.

Reference

FilePtr qi::openLocalFile(const qi::Path& localPath)

Brief:

Parameters:
  • localPath – Path to a file on the local file system that can be open.
Returns:

A shareable access to the opened local file.

Open a local file located at the specified path and provide it for reading as a sharable file access.

FutureSync<void> qi::copyToLocal(FilePtr file, Path localPath)

Brief:

Parameters:
  • file – Source file to copy.
  • localPath – Local file system location where the specified file will be copied. No file or directory should be located at this path otherwise the operation will fail.
Returns:

A synchronous future associated with the operation.

Copy an open local or remote file to a local file system location.

ProgressNotifierPtr qi::createProgressNotifier(Future<void> operationFuture = {})

Brief:

Parameters:
  • operationFuture – Optional future of an operation to associate the notifier with.
Returns:

A progress notifier, associated to the operation of the future if provided.

Create and provide a remotely shareable ProgressNotifier object.

typedef qi::Object< File > FilePtr
Pointer to a file with shared/remote semantic.
typedef Object< FileOperation > FileOperationPtr
Pointer to a file operation with sharing semantic.
typedef qi::Object< ProgressNotifier > ProgressNotifierPtr
Pointer to a ProgressNotifier with shared/remote semantic.

qi::File Class Reference

Introduction

More...

#include <qicore/file.hpp>

Public Members

const std::streamsize MAX_READ_SIZE

Public Functions

()
std::streamsize size() const
bool isOpen() const
bool isRemote() const
ProgressNotifierPtr operationProgress() const
Buffer read(std::streamsize countBytesToRead)
Buffer read(std::streamoff beginOffset, std::streamsize countBytesToRead)
bool seek(std::streamoff offsetFromBegin)
void close()

Detailed Description

Provide access to the content of a local or remote file. Should be obtained using openLocalFile() or through a service API if the file is potentially remote.

Members Documentation

const std::streamsize qi::File::MAX_READ_SIZE

Maximum count of bytes that you can read by reading functions call.

Function Documentation

()
std::streamsize qi::File::size() const = 0

Brief:

Returns:Total count of bytes contained in the file or 0 if the file is closed.
bool qi::File::isOpen() const = 0

Brief:

Returns:true if the file is currently open for reading, false otherwise.
bool qi::File::isRemote() const = 0

Brief:

Returns:true if the file is located on a remote filesystem, false otherwise.
ProgressNotifierPtr qi::File::operationProgress() const = 0

Provide the progress notifier used by the operations manipulating this file. The notifier is associated with this file. Therefore, no concurrent operation should be used by this notifier object, as it is not safe to have concurrent operations running on the same file.

Buffer qi::File::read(std::streamsize countBytesToRead) = 0

Brief:

Parameters:
  • countBytesToRead – Count of bytes to read from the file, starting from the current position of the file cursor.
Returns:

A buffer of data read from the file, empty if there is no data in the specified byte range to read or if the file have been closed. If there is less data to read in the file than the required count, if we try reading past the end of the file for example, then the buffer will only contain the available data, nothing more.

Read a specified count of bytes starting from the current cursor position.

Buffer qi::File::read(std::streamoff beginOffset, std::streamsize countBytesToRead) = 0

Brief:

Parameters:
  • beginOffset – Position in the file to start reading from.
  • countBytesToRead – Count of bytes to read from the file starting at the current position of the file cursor.
Returns:

A buffer of data read from the file, empty if: there is no data in the specified byte range to readif the file have been closed;if the start position is outside the available range of data in the file. If there is less data to read in the file than the required count, if we try reading past the end of the file for example, then the buffer will only contain the available data, nothing more

Read a specified count of bytes starting from a specified byte position in the file.

bool qi::File::seek(std::streamoff offsetFromBegin) = 0

Brief:

Parameters:
  • offsetFromBegin – New position of the read cursor in the file. If it is out of the range of data in the file, the cursor will not be changed at all.
Returns:

true if the position is in the range of data available in the file, false otherwise, in which case the cursor have not been changed.

Move the read cursor to the specified position in the file.

void qi::File::close() = 0

Close the file. Once this function is called, calling most other operation will throw a std::runtime_error. The size(), isOpen() and isRemote() calls will return work as expected.

qi::ProgressNotifier Class Reference

Introduction

More...

#include <qicore/file.hpp>

Enumerations

enum Status
Name Brief
Status_Idle The operation has not start yet.
Status_Running The operation is currently running.
Status_Finished The operation finished successfully.
Status_Failed The operation has failed.
Status_Canceled The operation has been canceled by the user.

Public Members

Property<Status> status
Property<double> progress

Public Functions

()
bool isRunning() const
Future<void> waitForFinished()
void reset()
void notifyRunning()
void notifyFinished()
void notifyCanceled()
void notifyFailed()
void notifyProgressed(double newProgress)
()
()
()
()
()
()

Detailed Description

Provide information about the state of a potentially long remote or async operation.

Members Documentation

Property<Status> qi::ProgressNotifier::status

Current status of the operation associated to this notifier. ProgressNotifier::Status

Property<double> qi::ProgressNotifier::progress

Progress state of the operation associated with this notifier. By default you can assume a normalized value ranging between 0.0 (no work is done) to 1.0 (all work is done). The semantic of this value is defined by the operation implementation and could be different from the default but should then be documented.

Function Documentation

()
bool qi::ProgressNotifier::isRunning() const = 0

Brief:

Returns:true if the operation associated to this notifier has started and is neither finished nor canceled nor failed yet, false otherwise.
Future<void> qi::ProgressNotifier::waitForFinished() = 0
void qi::ProgressNotifier::reset() = 0

Reset the status of the algorithm to the idle state with no progress.

void qi::ProgressNotifier::notifyRunning() = 0

Notify the observers that the operation associated with this notifier is now running.

void qi::ProgressNotifier::notifyFinished() = 0

Notify the observers that the operation has successfully ended.

void qi::ProgressNotifier::notifyCanceled() = 0

Notify the observers that the operation has been canceled by the user.

void qi::ProgressNotifier::notifyFailed() = 0

Notify the observers that the operation has failed.

void qi::ProgressNotifier::notifyProgressed(double newProgress) = 0

Brief:

Parameters:
  • newProgress – New value representing the total progress of the operation. By default, uses a range from 0.0 (no work has been done yet) to 1.0 (all work is done). The operation implementation is free to use another range if necessary but should clarify the meaning of this value in its documentation.

Notify the observers that the operation progressed.

()
()
()
()
()
()

qi::FileOperation Class Reference

Introduction

More...

#include <qicore/file.hpp>

Public Functions

~FileOperation()
()
()
FileOperation(FileOperation&& other)
FileOperation& operator=(FileOperation&& other)
qi::Future<void> start()
qi::Future<void> detach()
auto operator()()
ProgressNotifierPtr notifier() const
bool isValid() const
operator bool() const

Types

typedef boost::shared_ptr< Task > TaskPtr

Detailed Description

Base type for file operation exposing information about its progress state. Exposes a ProgressNotifier, associated to the operation.

Function Documentation

qi::FileOperation::~FileOperation()

Destructor. Cancel the operations’s task if is still running and this object is valid.

()
()
qi::FileOperation::FileOperation(FileOperation&& other)

Brief:

Parameters:
  • other – Object that will be moved-from. Will be in invalid state after this call, until being assigned to a vallid state.

Move construction.

FileOperation& qi::FileOperation::operator=(FileOperation&& other)

Brief:

Parameters:
  • other – Object that will be moved-from. Will be in invalid state after this call, until being assigned to a vallid state.

Move assignation.

qi::Future<void> qi::FileOperation::start()

Brief:

Returns:A future corresponding to the end of the operation.

Starts the operation’s task. This function must be called only once. Throws a std::runtime_error if start() has already been called before at least once or if this object is in an invalid state.

qi::Future<void> qi::FileOperation::detach()

Brief:

Returns:A future corresponding to the end of the operation.

Detach the running operation from this object. Useful to dissociate the on-going operation from the lifetime of the object, in order to allow its continuation after object destruction. The object destructor will cancel any still running operation if not dissociated beforehand.

Once called, this object will be in invalid state. The task must have been started before calling this function, otherwise a std::runtime_exception will be thrown.

auto qi::FileOperation::operator()

Call operator: calls start()

ProgressNotifierPtr qi::FileOperation::notifier() const

Brief:

Returns:A progress notifier associated to the operation if the operation’s task is owned and this object is valid, null otherwise.
bool qi::FileOperation::isValid() const

Brief:

Returns:True if this object is in a valid state, false otherwise. In an invalid state, all of this object’s member function calls will result in exception thrown except validity checks functions and move-assignation. An invalid object can be re-assigned to a valid state.
qi::FileOperation::operator bool() const

Brief:

Returns:True if this object owns the operation’s task, false otherwise.

qi::FileCopyToLocal Class Reference

Introduction

More...

Public Functions

FileCopyToLocal(qi::FilePtr file, qi::Path localPath)

Detailed Description

Copies a potentially remote file to the local file system.

Function Documentation

qi::FileCopyToLocal::FileCopyToLocal(qi::FilePtr file, qi::Path localPath)

Brief:

Parameters:
  • file – Access to a potentially remote file to copy to the local file system.
  • localPath – Local file system location where the specified file will be copied. No file or directory should be located at this path otherwise the operation will fail.

Constructor.

« qi::Url | C++ qi API Reference | C++ Class Index »
« previous | up | next »