Batch File: Split Compressed Archive based on Folders not size

I have had a batch file that would use 7zip to compress and encrypt an archive. The folders that were compressed contained millions of files, and I had already moved to the 64 bit version of 7zip to get me around a previous limit. Whilst this has been working fine for months, the folders and files are continuously growing and the other day it encountered an error stating

“System error:
Not enough storage is available to process this command.”

Now this error is not related to disk space but I believe its related to server resources. I did some initial research and I found an article about IRPstack size which looks interesting as to potentially fixing this, however the archive itself was becoming unmanageable so I decided to create a split archive to create smaller archives.

Now normally you can just use the split archive functionality built into 7zip, however this works on size and creates 1 archive split into smaller files. This was not what I wanted. I wanted to split the archive based on folder structure.

C:\rootfolder

\folder1
\folder2
\folder3

I wanted to create archive for each of the folders within the rootfolder.

So I came up with a batch file that loops through the rootfolder and takes each folder and runs the archive on that folder.

Before getting into the code I have this batch called from another batch file that sets up some standard variables, like dates and program locations.

%c_DateYYYYMMDD%
%c_TimeHHMM%
"%c_7zipFolder%

@echo on
SETLOCAL ENABLEDELAYEDEXPANSION
title %c_jobName%

SET sSourceFolder=C:\rootfolder
SET sZipFilesArchiveFolder=archive\files
SET sZipFilesArchiveFileName=archivefilename

SET sRemoteServer=remoteserver1
SET sRemoteDrive=D

echo on

REM Remove the double quotes from the front and end of the root path
REM REMOVE quotes using string substiution
SET sSourceFolder=%sSourceFolder:"=%

FOR /F "DELIMS==" %%d in ('DIR "%sSourceFolder%" /AD /B') DO (

REM Set up filename for archive

SET c_ZipFilesArchive=%c_DateYYYYMMDD%_%c_TimeHHMM%_%ComputerName%_%sZipFilesArchiveFileName%_%%d_X64.7z

REM Run 7zip

"%c_7zipFolder%\7z" a -t7z -mhe=on -mhc=on -w"\\%sRemoteServer%\%sRemoteDrive%$\%sZipFilesArchiveFolder%" -p"password1" "\\%sRemoteServer%\%sRemoteDrive%$\%sZipFilesArchiveFolder%\!c_ZipFilesArchive!" "%sSourceFolder%\%%d"> “.\7zipoutput.log"

)
ENDLOCAL

So the batch file is pretty simple, but things to note are you have to use delayed expansion with the for loop to set the variable contents (hence the variable surrounded in !! and not %%, in addition the SET ENABLEDELAYEDEXPANSION at the beginning.

The script loops through the folders returned by the DIR /AD /B command, sets %%d to the folder name and that is used within the for loop to name the archive and archive the contents.

The script sets the working file for the compression to the remote location.

References

Error message: "Not enough server storage is available to process this command"

Not enough server storage

Batch File: Split Compressed Archive based on Folders not size