How to store lot of read-only data in DLL?
I need a way to store lot of numerical read-only data (double values) in a DLL. Data must be embedded in source, because it will be changed from time to time - no external tools like Visual Studio etc., only code generation+shell compiler.
I know that it is a bad practice, but this is "business requirement".
I need to store about 2 000 000 of 32-bit float values. This should be about 8MB raw data. Assume that DLL size limit is 100MB. Language is Fortran.
My first thought was to generate value assignments to dynamic array:
work(1,1,1,1) = 826 work(1,1,1,2) = 935 work(1,1,1,3) = 712.5 work(1,1,1,4) = 617.1 work(1,1,1,5) = 102.2
But final source file has about 70 megabytes, and 0 chances to compile (compiler out of memory error). Besides, even if it would compile, final DLL have size of ~5MB for each 0.5MB of useful data.
Any ideas to format source in more compact way (compiler optimisation friendly?) or maybe pack data to other structure like string/raw binary data and extract on runtime ?
This is exactly the kind of use the DATA statement has:
real work(10, 10, 10, 10) data work/826, 935, 712.5, 617.1, 102.2/
Depending on which compiler you are using, you can even make the data readonly so that any attempt by the program to change any bit of it results in a segfault exception.
Some compilers support the explicit array member enumeration in the data statement:
data work(1,1,1,1)/826/, work(1,1,1,2)/935/, work(1,1,1,3)/712.5/ data work(1,1,1,4)/617.1/, work(1,1,1,5)/102.2/
which will be easier to generate in an unordered way.
I do not know if there is a better solution using Fortran but if everything else fails, you could do the following.
Just treat your 2,000,000 32 bit floats as 8,000,000 bytes and Base64 encode them into a string. You will end up with a 12,800,000 character long string and you could or probably even should split it into several parts. At runtime just reverse everything.