Append to an OS file in basic+?

vince · January 2023

Does Basic+ have a built-in way to append data to an OS file, something akin to opening a file in "a" mode in Python, or even >> in the Bourne shell?

While one can read the contents of the file into a variable with OSRead, append to the variable and then write the value back to the file with OSWrite, I notice that OSBWrite could almost be used to append. The documentation mentions that with byte equal to 0, writing starts at the beginning of the file. It would be handy if some other special value for byte would start writing at the end of the file, but the documentation doesn't go so far to say, and I haven't found that special value. Has anyone by some remote chance discovered an undocumented append feature of OSBWrite?

AusMarkB · January 2023

I don't know of one but...

I was trying to think of a reason/use case why you wouldn't just use the OSRead/Write approach you mentioned and the only thing I could think of was that you want to do multiple appends within a loop, to a file which likely already has data before the first append. I assume something like a log file that doesn't get cleared automatically.

If I'm on the right track, then you could use a combination of OSRead and then OSBWrite.
Upon initial opening of the file, OSRead the contents into a variable, use the GetByteSize Function to calculate the number of bytes and then begin your OSBWrites from there instead of zero.

DonBakke · January 2023

@vince - Mark's answer is the right one.

vince · January 2023

My motivation is partly more efficient programs. Reading and writing large files would take longer. And then I am influenced by what I have found in other programming languages. But your approach to avoid writing the entire file contents back to file is helpful, and I can use that. Thank you.

DonBakke · January 2023

You can use the Dir() function to get the size of the file and avoid having to OSRead (or use OSBRead and start where you want).

Opto_Will · January 2023

What are the efficencies in using Dir() or GetByteSize in a OSRead?

When logging to OS I normally just do something quick and easy like:

declare function Get_File_Format
 
$Insert APP_INSERTS     ; // To get CRLF$ and TAB$
 
SaveTextFile = 1        ; // Normally set at Top of Procedure as a toggle when debugging or through parameter.  
                          // This allows many log entries in a script to be easily enabled/disabled.
 
If SaveTextFile = 1 Then
    FileFormat  = Get_File_Format()
    Dos.File    = FileFormat:"_My_Log_File.txt"     ;    // Change this as required.
    TimeStamp   = TimeDate()
             
    // Make output nicer to read.
    Method_Fmt      = fmt(Method,"L#25")
    URL_Fmt         = fmt(URL,"L#50")
    HeaderList_Fmt  = fmt(HeaderList,"L#25")
    Body_Fmt        = fmt(Body,"L#25")
 
    // Build Output text. Adjust as needed.
    MessageText  = TimeStamp : CRLF$ : "METHOD:" : Method_Fmt : CRLF$ : "URL:" : URL_Fmt : CRLF$
    MessageText := "HEADER:" : HeaderList_Fmt : CRLF$ : "BODY:" : Body_Fmt : CRLF$ : "TOKEN:"
    MessageText := OAuthAccessToken@
     
    //Get rid of MultiValue delimiters from variables. Adj as needed
    Swap @VM with TAB$ in MessageText
    Swap @FM with CRLF$ in MessageText
    Swap @TM with CRLF$ in MessageText
     
    // If exists, Append.
    OSRead CurrentData From Dos.File Then
        CurrentData = CurrentData:CRLF$:"______________________________":CRLF$:CRLF$:MessageText
    End Else
        CurrentData = MessageText
    End
                 
    OSWrite CurrentData To Dos.File
end

Get_File_Format is just a little procedure to get a path and start a datestamp as the filename prefix like:
C:\MyLogs\230124
so my file ends up a consistent structure:
C:\MyLogs\230124_My_Log_File.txt

This keeps me a running log daily that appends the info I am after and I can look up historically if I have those logs automated.

I am always interested in better ways.

DonBakke · January 2023

Your standard approach is just fine for files that aren't expected to grow significantly (and don't ask me what qualifies as "significant"). Just note that each time you read the file you are doing memory management and incurring overhead. As local variables get to be a certain size you'll notice some performance degradation within OI.

I don't know how performant the Dir() function is, but I'm fairly convinced it relies upon a Windows API to return the data thus saving the need to read the content. Therefore, I would expect it to perform favorably and uniformly regardless of the size of the file.

Opto_Will · January 2023

I get that @DonBakke.

To rudimentarily reiterate what you and @AusMarkB indicated I could replace my OSREAD/OSWRITE with something like:

OSOpen FullyQualifiedFileName To filevar then
    CurrByteMark = Dir(FullyQualifiedFileName)<1>
    OSBWrite LogMessage to filevar at CurrByteMark + 1
End

And this should be more beneficial the more the log grows...

AusMarkB · January 2023

@Opto_Will, you could

To rudimentarily reiterate what you and @AusMarkB indicated I could replace my OSREAD/OSWRITE with something like:

OSOpen FullyQualifiedFileName To filevar then
CurrByteMark = Dir(FullyQualifiedFileName)<1>
OSBWrite LogMessage to filevar at CurrByteMark + 1
End

but you'd also be correct in wondering at what point that would become more efficient and therefore what's the benefit?

I was more suggesting a scenario like

// closing application
OSOpen FullyQualifiedFileName to filevar then
    CurrByteMark = Dir(FullyQualifiedFileName)<1>
    theSelect   = "SELECT SOMETABLE WITH DATE = ":Quote(Oconv(today, "DE"))
    RList(theSelect, 5, "", "", "")
    eof = false4
    Loop
        ReadNext tableID else eof = true$
    Until eof do
        Read logRec from f_someTable, tableID then
            dosText = logRec<1>:" ":logRec<2>:" at ":logrec<4>:crlf$
            dosBytes    = GetByteSize(dosText)
            OSBWrite dosText to filevar at CurrByteMark
            CurrByteMark    += dosBytes
        end
    Repeat
end

or perhaps a background function that does something similar, keeps the filevar and currbytemark in common and writes a row away each time a specific event or various events occur.

For a one off write, I'd do the same as you, read, append to variable, write back.

Opto_Will · January 2023

Ahh. I see your Use Case now.
Good options for everyone!

DonBakke · January 2023

We wrote a Logging_Services module to encapsulate the above approach. The two most prominent services are NewLog and AppendLog. The NewLog service primarily initializes the file and allows you to define column headers. It has an optional flag to keep the content in the file if it already exists. This service then returns a handle that you can pass into the AppendLog service. The AppendLog service works very much like Mark's code above.

Opto_Will · January 2023

So you did @ DonBakke.
Wiki documentation and everything!

I knew that as I have seen that in SRP code but for some reason I keep forgetting some of the helpful things I come accross...

DonBakke · January 2023

Wiki documentation and everything!

LOL. I totally forgot that this is included with the HTTP Framework and thus documented in the wiki!

Opto_Will · January 2023

So not just me forgetting stuff!
I feel a little better now ;-)

Append to an OS file in basic+?

Comments