ARTICLE AD BOX
I am working on a device where the I/O is incredibly slow and I have to SHA256 hash some data. So, I thought that I would use a double buffer to at least be able to do some I/O whilst the hash is being processed.
The process is to read into buffer one, swap buffers, hash buffer two on a background thread and the I/O will start again on buffer one. It's not going to gain me a significant advantage being that the hashing is super fast and the I/O really slow, but any I/O that I can do whilst the hash it generated, is going to help.
Now I've written the below and it works... However, I would like to know if:
There are any improvements that can be made. There are probably loads.
Glaring omissions or just plain a simple bugs or race conditions.
Whether there is anything in C++20 that can help, maybe some atomics.
Would it be better to have a thread for reading and one for processing? Would that make the solution easier?
Would it even be better to have the I/O on the thread and the processing on the main thread?
This is quite a good answer, but it's not quite what I was after: Multithreaded double buffer
bool Ready = false, Quit = false, Started = false; std::mutex ReadMtx, StartMtx; std::condition_variable Started_CV, Process_CV; static constexpr size_t BlockSize = static_cast<size_t>(1 << 24); std::unique_ptr<uint8_t[]> FileBlockOne(new uint8_t[BlockSize]); std::unique_ptr<uint8_t[]> FileBlockTwo(new uint8_t[BlockSize]); #pragma pack(push) #pragma pack(1) typedef struct { uint8_t Hash[SHA256::SHA256_HASH_SIZE]; uint64_t FileSize; uint64_t CreateTime; uint16_t Flags; } HashFile; #pragma pack(pop) HashFile FileHash{}; void CreateHash(const std::string& file_path) { sha256.SHA256_Init(); std::thread HashThread(UpdateHash); const size_t file_sz = std::filesystem::file_size(file_path); size_t count = file_sz >> 24; std::fstream fs(file_path, std::ios::in | std::ios::binary); for (size_t i = 0; i < count; i++) { fs.read(reinterpret_cast<char*>(FileBlockOne.get()), BlockSize); { std::scoped_lock<std::mutex> lk(ReadMtx); // Get the read Mutex and lock it for the buffer swap FileBlockOne.swap(FileBlockTwo); Ready = true; Process_CV.notify_one(); // Notify waiting thread that it has data } std::unique_lock<std::mutex> lock(StartMtx); // Wait for the notified thread to start processing and Started_CV.wait(lock, [this]() { return Started; }); // notify the main thread when it has done Started = false; } Quit = true; Process_CV.notify_one(); // Notify the thread that it should die HashThread.join(); // Wait for it to complete count = (file_sz - (count << 24)); fs.read(reinterpret_cast<char*>(FileBlockOne.get()), count); sha256.SHA256_Update(FileBlockOne.get(), count); sha256.SHA256_Final(FileHash.Hash); fs.close(); } void UpdateHash(void) { for (;;) { std::unique_lock<std::mutex> lock(ReadMtx); // Wait until data is ready for this thread Process_CV.wait(lock, [this]() { return Ready || Quit; }); Ready = false; if (Quit) break; { std::scoped_lock<std::mutex> lk(StartMtx); Started = true; Started_CV.notify_one(); // Notify main thread that it can resume as this thread has } sha256.SHA256_Update(FileBlockTwo.get(), BlockSize); // started } }