oo2c is the Optimizing Oberon-2 Compiler from the OOC project: an
Oberon-2 to C translator with an optimizer built on GSA (guarded
single-assignment form), a large standard library, and tools such as
the oocdoc documentation generator. Its development stopped with
release 2.1.11 in 2011, and that release no longer builds on current
systems.
My fork brings it back. Version 2.2.4 builds, runs and passes its tests
on:
Linux with current gcc and Boehm gc: Fedora 44 and 45 and Ubuntu
26.04, on x86-64, aarch64 and i386;
FreeBSD 15.1 (x86-64 and aarch64);
OpenBSD 7.9 and NetBSD 11.0, both 64-bit (amd64) and 32-bit (i386),
and NetBSD 11.1.
GitHub's CI builds, self-hosts and tests it on Linux and in FreeBSD,
OpenBSD and NetBSD virtual machines. A release tarball bootstraps with
no oo2c installed: ./configure && make && make install. There is
oo2c_64 for 64-bit systems and oo2c_32 for 32-bit ones, signed,
with SHA-256 sums.
What changed since 2.1.11
The language. Using a nested procedure as a procedure value is
now an error, as the Oberon-2 report requires; the generated C would
have needed an executable stack. Programs that did this must be
changed. A POINTER [CSTRING] TO ARRAY OF CHAR no longer allows
LEN(p^), which read memory before the string.
The generated C compiles with gcc 16 with no warnings, and the
compiler and library build with no warnings.
Checking.make test-valgrind and make test-asan run the
test programs under valgrind, and under AddressSanitizer and
UndefinedBehaviorSanitizer. Undefined behaviour they found in
ASH, SYSTEM.ROT, empty open arrays and the hash functions is
fixed.
Threads (--enable-threads=pthreads) work on Linux and the
three BSDs.
The build works with autoconf 2.72 and libtool 2.5, and make
dist and make dist32 work again.
NEWS lists every change, release by release, and INSTALL the
packages and configure settings for each system.
oo2c is licensed as upstream: the compiler under the GNU GPL version 2,
the library under the GNU LGPL.
oo2c sits beside my own Oberon-2 compiler, Peaseblossom, which I
announced earlier today.
Releases, bug reports and the source are on GitHub. Comments, bug
reports and patches are welcome.
I'm announcing a set of Oberon-2 projects: a compiler, libraries, and
programs built with them. All are on GitHub, and the software is under
the BSD 3-Clause License.
For voc (Vishap Oberon)
olibfyaml: a binding to libfyaml, the YAML 1.2 parser and
emitter. It covers parsing, navigation, typed values, building and
emitting YAML or JSON, anchors, tags and source locations. Errors in
the input come back as values with file:line:column messages.
For both voc and poc
opp: the Oberon pretty printer. It reformats modules within a
maximum line length and keeps every comment. It handles both
compilers' language extensions. Before writing, it checks that its
output has exactly the tokens and comments of its input.
Ropes: ropes, strings represented as trees of pieces, with a
self-test and a command-line demo.
ArgParser: command-line parsing: long and short options,
clustered short options, option values, commands, and usage messages.
Peaseblossom
poc, the Peaseblossom Oberon Compiler: an Oberon-2 compiler with
an LLVM backend for Linux, FreeBSD, OpenBSD and NetBSD, and a VAX
backend for VAX/VMS 5.5-2. It was motivated in large part by my wish
for a more modern language than the limited ones available on the VAX
system I maintain. A release tarball needs only clang and GNU make,
or it can be bootstrapped using Vishap Oberon. Packages are available
for Fedora, Debian, FreeBSD, OpenBSD and NetBSD. It comes with a
User's Guide and a Reference Guide.
polibfyaml: olibfyaml for poc, with the same API, tests and
examples.
ova_fmt_po, an example program using polibfyaml: it formats
characters for the OVA tabletop RPG from YAML as reStructuredText,
gemtext, Markdown and more.
pofltk: a thick binding to FLTK 1.4. FLTK's classes are record
types a program extends, and draw() and handle() are
type-bound procedures it overrides. Coverage includes widgets, layout
(flex and grid), menus, dialogs, text editors, browsers, trees,
tables, images, the clipboard and preferences. FlTest drives a
program's windows in tests on Xvfb or a headless compositor. Many of
FLTK's test programs are ported as examples.
ova_cg_po, an example program using pofltk: a character
generator for OVA. It's a complete program with a menu bar, tabs,
grids, file choosers, preferences and window tests. Its tour for
pofltk readers says where in its code each technique is.
poDateTime: dates, times, durations and time zones. It has
instants, the proleptic Gregorian calendar, clocks, POSIX TZ rules
and tz database files, ISO 8601, RFC 3339 and strftime formats. It
runs on the four Unixes and on VAX/VMS.
pounix: Unix services that poc's runtime doesn't provide:
dropping root for good, file descriptors and directories, running
child processes with an argument list (no shell), and errno.
posockets: TCP (IPv4 and IPv6) and Unix-domain sockets. It works
with poc's cooperative Tasks, so one thread can serve many
connections, with deadlines.
polibtls: a binding to libtls (LibreTLS on Linux, FreeBSD and
NetBSD; OpenBSD's own). It supports server contexts with several key
pairs, client certificates, and non-blocking I/O with Tasks. It's
tested against openssl, gnutls-cli and Go's TLS.
pogemini: a Gemini server built on the four libraries above. It
serves several capsules by host name, each with its own certificate,
and can require client certificates for parts of a capsule. Every
connection runs in a task in one thread.
poDB: a database-independent SQL interface, with drivers for
SQLite and PostgreSQL. A program moves from one to the other by
changing its connection URL and the driver it imports.
Comments, bug reports and patches are welcome. Many of the poc
libraries are early in their lives; polibfyaml and pofltk have had the
most use.
I updated the NCURSES PROGRAMMING HOWTO from
The Linux Documentation Project
slightly so the example programs would compile with modern compilers
and my pull request was accepted in January 2025, but unfortunately
the updated version has not made it online at TLDP.org. So I decided
to put it online here:
The Ada Quality and Style: Guidelines for Professional
Programmers is available on the web, but I don't think that there is
a PDF copy anywhere easily found, so I'm making a copy available
here. This is the style guide for Ada 83, by the way. I got it from
the Ada Information Clearinghouse. I downloaded the file
style-ps.zip
into a temporary directory and then ran the shell script
make-83style-pdf.sh to create the PDF file:
#! /usr/bin/env bash# First, unzip style-ps.zip. Then, in another directory, unzip all six of# those zip files. Then do the following:set-x
mkdirtmp&&cdtmp&&unzip-L../style-ps.zip&&mkdirtmp&&cdtmp&&foriin../*.zip;dounzip-L$i;done&&foriin*.ps;dops2pdf$i$(basename$i.ps).pdf;done&&qpdf--empty-pagescovers.pdf1-3--covers_fixed.pdf&&pdfunitecovers_fixed.pdfpreface.pdfack.pdftoc.pdfintro.pdfsource_c.pdfreadabil.pdfprog_str.pdfprog_pra.pdfconcurr.pdfportabil.pdfreuse.pdfperforma.pdfcomplete.pdfappendix.pdfref.pdfbib_0188.pdfindex.pdf83style.pdf
Note the qpdf command? That pulls out the first three pages
of covers.pdf, because the other pages duplicate stuff that is in
the other .pdf files.
Then I read through the PDF and made a bookmarks file for use with
jpdfbookmarks (I use one of the releases from this github repo):
Ada Quality and Style/1
Cover 1/1
Cover 2/2
Preface/4
Authors and Acknowledgements/6
Contents/8
Chapter 1: Introduction/10
1.1 How to use this book/11
1.2 To the new Ada programmer/11
1.3 To the experienced Ada programmer/12
1.4 To the software project manager/12
1.5 to contracting agencies and standards organizations/13
Chapter 2: Source Code Presentation/14
2.1 Code formatting/14
2.1.1 Horizontal spacing/14
2.1.2 Indentation/15
2.1.3 Alignment of operators/18
2.1.4 Alignment of declarations/19
2.1.5 More on alignmeent/20
2.1.6 Blank lines/21
2.1.7 Pagination/22
2.1.8 Number of statements per line/23
2.1.9 Source code line length/24
2.2 Summary/24
Chapter 3: Readability/26
3.1 Spelling/26
3.1.1 Use of underscores/26
3.1.2 Numbers/26
3.1.3 Capitalization/27
3.1.4 Abbreviations/28
3.2 Naming conventions/29
3.2.1 Names/29
3.2.2 Type names/29
3.2.3 Object names/30
3.2.4 Program unit names/31
3.2.5 Constants and named numbers/32
3.3 Comments/33
3.3.1 General comments/34
3.3.2 File headers/35
3.3.3 Program unit specification header/35
3.3.4 Program unit body header/38
3.3.5 Data comments/39
3.3.6 Statement comments/41
3.3.7 Marker comments/43
3.4 Using types/44
3.4.1 Declaring types/44
3.4.2 Enumeration types/45
3.5 Summary/46
Chapter 4: Program Structure/50
4.1 High level structure/50
4.1.1 Separate compilation capabilities/50
4.1.2 Subprograms/51
4.1.3 Functions/52
4.1.4 Packages/53
4.1.5 Cohesion/53
4.1.6 Data coupling/54
4.1.7 Tasks/54
4.2 Visibility/55
4.2.1 Minimization of interfaces/55
4.2.2 Nested packages/56
4.2.3 Restricting visibility/56
4.2.4 Hiding tasks/57
4.3 Exceptions/59
4.3.1 Using exceptions to help define an abstraction/59
4.4 Summary/61
Chapter 5: Programming Practices/64
5.1 Optional parts of the syntax/64
5.1.1 Loop names/64
5.1.2 Block names/65
5.1.3 Exit statements/66
5.1.4 Naming end statemnts/66
5.2 Parameter lists/67
5.2.1 Formal parameters/67
5.2.2 Named association/67
5.2.33 Default parameters/68
5.2.4 Mode indication/69
5.3 Types/69
5.3.1 Derived types and subtypes/69
5.3.2 Anonymous types/71
5.3.3 Private types/71
5.4 Data structures/72
5.4.1 Heeterogeneous data/72
5.4.2 Nested records/73
5.4.3 Dynamic data/74
5.5 Expressions/75
5.5.1 Range values/75
5.5.2 Array attributes/76
5.5.3 Parenthetical expressions/76
5.5.4 Positive forms of logic/77
5.5.5 Short circuit forms of the logical operators/77
5.5.6 Accuracy of operations with real operands/78
5.6 Statements/78
5.6.1 Nesting/78
5.6.2 Slices/80
5.6.3 Case statements/80
5.6.4 Loops/81
5.6.5 Exit statements/82
5.6.6 Recursion and iteration bounds/83
5.6.8 Return statements/84
5.6.9 Blocks/85
5.6.10 Aggregates/86
5.7 Visibility/86
5.7.1 The use clause/87
5.7.2 The renames clause/88
5.7.3 Overloaded subprograms/88
5.7.4 Overloaded operators/89
5.7.5 Overloading the equality operator/89
5.8 Using exceptions/89
5.8.1 Handling versus avoiding exceptions/90
5.8.2 Handlers for others/90
5.8.3 Propagation/91
5.8.4 Localizing the causes of an exception/91
5.9 Erroneous Execution/92
5.9.1 Unchecked conversion/92
5.9.2 Unchecked deallocation/93
5.9.3 Dependeence on parameter passing mechanism/93
5.9.4 Multiple address clauses/94
5.9.5 Suppression of exception check/94
5.9.6 Initialization/94
5.9.7 Direct_IO and Sequential_IO/95
5.9.8 Incorrect order dependencies/96
5.10 Summary/96
Chapter 6: Concurrency/100
6.1 Tasking/100
6.1.1 Tasks/100
6.1.2 Anonymous task types/101
6.1.3 Dynamic tasks/102
6.1.4 Prioritis/103
6.1.5 Delay statements/104
6.2 Commmunication/105
6.2.1 Effecient task communications/105
6.2.2 Defensive task communication/106
6.2.3 Attributes 'Count, 'Callable, and 'Terminated/108
6.2.4 Shared variables/109
6.2.5 Tenative rendezvous constructs/111
6.2.6 Commmunication complexity/112
6.3 Termination/113
6.1.1 Avoiding termination/113
6.3.2 Normal termination/114
6.3.3 The abort statement/115
6.3.4 Abnormal termination/115
6.4 Summary/116
Chapter 7: Portability/118
7.1 Fundamentals/119
7.1.1 Global assumptions/119
7.1.2 Actual limits/120
7.1.3 Comments/120
7.1.4 Main subprogram/121
7.1.5 Encapsulating implementation dependencies/121
7.1.6 Implementation-added features/122
7.2 Numeric types and expressions/122
7.2.1 Predefined numeric types/122
7.2.2 Ada model/123
7.2.3 Analysis/123
7.2.4 Accuracy Constraints/124
7.2.5 Comments/124
7.2.6 Precision of constants/124
7.2.7 Subexpression evaluation/124
7.2.8 Relational tests/124
7.3 Storage control/125
7.3.1 Representation clause/125
7.4 Tasking/126
7.4.1 Task activation order/126
7.4.2 Delay statements/126
7.4.3 Package Calendar, type Duration, and System.Tick/126
7.4.4 Select statement evaluation ordr/126
7.4.5 Task scheduling algorithm/127
7.4.6 Abort/127
7.4.7 Shared variables and pragma shared/127
7.5 Exceptions/127
7.5.1 Predefined exceptions/127
7.5.2 Constraint_Error and Numeric_Error/128
7.5.3 Implementation-defined exceptions/128
7.6 Representationi clauses and implementation dependent features/128
7.6.1 Representation clauses/129
7.6.2 Package System/129
7.6.3 Machine code inserts/129
7.6.4 Interfacing Foreign Languages/130
7.6.5 Implementation-defined pragmas and attributes/130
7.6.6 Unchecked deallocation/130
7.6.7 Unchecked conversion/131
7.7 Input/output/131
7.7.1 Name and Form parameters/132
7.7.2 File closing/132
7.7.3 I/O on access types/132
7.7.4 Package Low_Level_IO/132
7.8 Summmary/133
Chapter 8: Reusability/136
8.1 Understanding and clarity/137
8.1.1 Application-independent naming/137
8.1.2 Abbreviations/138
8.1.3 Generic formal parameters/139
8.2 Robustness/139
8.2.1 Named numbers/139
8.2.2 Unconstrained arrays/140
8.2.3 Assumptions/140
8.2.4 Subtypes in generic specifications/141
8.2.5 Overloading in generic units/143
8.2.6 Hidden tasks/143
8.2.7 Exceptions/143
8.3 Adaptability/145
8.3.1 Complete functionality/145
8.3.2 Generic units/146
8.3.3 Using generic units to encapsulate algorithms/146
8.3.4 Using generic units for abstract data types/148
8.3.5 Iterators/151
8.3.6 Private and limited private types/154
8.4 Independence/156
8.4.1 Using generic parameters to reduce coupling/156
8.4.2 Coupling due to pragmas/157
8.4.3 Part families/158
8.4.4 Conditional compilation/158
8.4.5 Table-driven programming/159
8.5 Summary/160
Chapter 9: Performance/162
9.1 Improving exception speed/162
9.1.1 Pragma inline/162
9.1.2 Blocks/163
9.1.3 Arrays/163
9.1.4 Mod and rem operators/164
9.1.5 Constraint checking/164
9.2 Summmary/165
Chapter 10: Complete Examples/166
10.1 Menu-driven user interface/166
10.2 Line-oriented portablee dining philosophers example/174
10.3 Window-oriented portable dining philosophers example/179
Appendix A: Map from Ada Language Referencee Manual to Guidelines/188
References/194
Bibliography/198
Index/202
Right, So, I've talked about how the LBL Software Tools Virtual
Operating System organized its source files, using its archiver to
collect all the source files, etc. for a program into an archive
(sometimes with nested archives, because RECURSION IS NEAT) and then
putting the archive under revision control using the TCS commands
admin, delta, and get.
And that makes a lot of sense when what you are working on is one
program or another closely related collection of information. But
what if you are working on a collection of individual or nearly
individual programs? Or what if your program is large enough that
putting all its source in a single archive runs up against size limits
in the SWTOOLS programs? Remember, they were meant to run on the 16
bit minicomputers of the day, as well as the 32 bit minicomputers that
showed up soon after the SWTOOLS VOS was written, so there is a limit
on the number of lines in a file that these can handle somewhere
between 10,000 and 15,000 lines. What do you do then?
Well, today I wrote a set of commands to make it easy to work with a
collection of source files, which each have a corresponding TCS
history file in the subdirectory [.TCS]. I decided to have
one DCL command procedure whose first parameter was the command to
execute, so the common processing could be shared instead of
duplicated between programs. But I had already written a program to
compare a current source file inn the working directory with an older
version in a TCS revision file. Hmm.
Anyway, I went on to write the tcs command as a front end
for the normal TCS commands. Here's an example from running
TCS HELP
$ tcs help
usage:
tcs [ help | -h | -help | -? ]
Output this help message.
tcs admin FILE
Initialize TCS file [.TCS]FILE from FILE.
tcs get [-oOUTFILE] [-h] [-rM.N] FILE
Get a revision of FILE from TCS file [.TCS]FILE.
-oOUTFILE writes the output to OUTFILE.
If -o is not specifed the output goes to standard output.
-h Output history information instead of a revision.
-rM.N Output revision M.N.
tcs delta FILE
Add a new version to TCS file [.TCS]FILE from FILE.
tcs outdated
List all the source files that are newer than their corresponding
TCS file in [.TCS]
tcs uncovered [FILESPEC] [IGNOREDTYPES]
List all the files in the current directory that do not have
corresponding files in [.TCS].
If FILEPSEC is specified it should be a VMS wildcard that
will be applied to the current directory for matching.
If IGNOREDTYPES is specified it should be a single file type
(without the ".") or a comma separated list of file types
to be ignored.
If one needs to specified IGNOREDTYPES but not FILESPEC just
specify an empty string for FILESPEC.
And here is the current source code of TCS.SDCL:
!> TCS.SDCL -- TCS frontend.
##############################################################################
###
### tcs [ help | -h | -help | -? ]
### tcs admin FILE
### tcs get [-ooutfile] [-h] [-rM.N] FILE
### tcs delta FILE
### tcs outdated
### tcs uncovered [FILESPEC] [IGNOREDTYPES]
###
##############################################################################
wso :== write sys$output
wse :== write sys$error
TRUE = 1 .eq. 1
FALSE = 1 .ne. 1
QUIET = %x10000000
months_jan = "01"
months_feb = "02"
months_mar = "03"
months_apr = "04"
months_may = "05"
months_jun = "06"
months_jul = "07"
months_aug = "08"
months_sep = "09"
months_oct = "10"
months_nov = "11"
months_dec = "12"
debugging = f$type (tcs_debugging) .nes. ""
verbose = FALSE
command = p1
if (command .eqs. "HELP" .or.
command .eqs. "-H" .or.
command .eqs. "-HELP" .or.
command .eqs. "-?")
{
goto usage
}
if ((command .eqs. "") .or.
.not. (command .eqs. "ADMIN" .or.
command .eqs. "GET" .or.
command .eqs. "DELTA" .or.
command .eqs. "OUTDATED" .or.
command .eqs. "UNCOVERED"))
{
wse "tcs: unrecognized TCS command: """, command, """"
exit 2
}
param_idx = 2
num_options = 0
outfile = ""
while (f$extract (0, 1, p'param_idx') .eqs. "-") {
## Remember, DCL upcases everything!
if (f$extract (0, 2, p'param_idx') .eqs. "-O") {
outfile = f$extract (2, f$length (p'param_idx'), p'param_idx')
} else {
num_options = num_options + 1
options_'num_options' = p'param_idx'
}
param_idx = param_idx + 1
}
file = p'param_idx' # Can be blank
dirname = f$search ("TCS.DIR")
if (dirname .eqs. "") { # No [.TCS] directory.
create/dir [.TCS]
}
tcsfile = "[.tcs]" + file + "-TCS" # Not used by some commands.
if (debugging) {
wso "command: ", command, " file: ", file, " tcsfile: ", tcsfile
wso "outfile: """, outfile, """"
}
if (command .eqs. "ADMIN") {
goto do_admin
} else if (command .eqs. "GET") {
goto do_get
} else if (command .eqs. "DELTA") {
goto do_delta
} else if (command .eqs. "OUTDATED") {
goto do_outdated
} else if (command .eqs. "UNCOVERED") {
goto do_uncovered
}
wso "tcs: This should be impossible, so somebody screwed up. Exiting."
exit 2 .or. QUIET
do_admin:
result = f$search (tcsfile)
if (result .nes. "") {
wse "tcs: TCS file ", tcsfile, " already exists, exiting."
exit 2
}
file = f$edit (file, "LOWERCASE")
dclcmd = "admin ""-i''file'"" ''tcsfile'"
wso "Executing ", dclcmd
define/user sys$input sys$command
'dclcmd'
exit
do_get:
if (outfile .eqs. "") {
get 'options_1' 'options_2' 'options_3' 'options_4' 'tcsfile'
} else {
get 'options_1' 'options_2' 'options_3' 'options_4' 'tcsfile' >'outfile'
}
exit
do_delta:
result = f$search (tcsfile)
if (result .eqs. "") {
wse "tcs: TCS file ", tcsfile, " does not exist, exiting."
exit 2
}
define/user sys$input sys$command
delta 'file' 'tcsfile'
exit
do_outdated:
filespec = "[.tcs]*.*-tcs"
repeat {
tcsfile = f$search (filespec)
if (tcsfile .eqs. "") break
tcsrdt = f$file (tcsfile, "RDT")
tcs_day = f$extract (00, 02, tcsrdt)
tcs_monname = f$extract (03, 03, tcsrdt)
tcs_year = f$extract (07, 04, tcsrdt)
tcs_rest = f$extract (11, 12, tcsrdt)
tcs_monnum = months_'tcs_monname'
tcs_cmpdate = tcs_year + tcs_monnum + tcs_day + tcs_rest
if (debugging .and. verbose) wso "tcs_cmpdate: ", tcs_cmpdate
dirname = f$parse (tcsfile,,, "DIRECTORY") - ".TCS]" + "]"
filename = f$parse (tcsfile,,, "NAME")
filetype = f$parse (tcsfile,,, "TYPE") - "-TCS"
srcfile = filename + filetype
srcrdt = f$file (srcfile, "RDT")
src_day = f$extract (00, 02, srcrdt)
src_monname = f$extract (03, 03, srcrdt)
src_year = f$extract (07, 04, srcrdt)
src_rest = f$extract (11, 12, srcrdt)
src_monnum = months_'src_monname'
src_cmpdate = src_year + src_monnum + src_day + src_rest
if (debugging .and. verbose) wso "src_cmpdate: ", src_cmpdate
if (src_cmpdate .gts. tcs_cmpdate) wso srcfile, " is newer than ", tcsfile
}
exit
do_uncovered:
i = param_idx + 1
ignoredtypes = p'i'
if (ignoredtypes .nes. "") ignoredtypes = "," + ignoredtypes
if (debugging) wse "ignoredtypes: ", ignoredtypes
ignoredtypes_len = f$length (ignoredtypes)
if (file .eqs. "") filespec = "*.*"
else filespec = file
if (debugging) wse "filespec: ", filespec
old_result = ""
repeat {
result = f$search (filespec, 1)
if (debugging) wso "result: ", result
# No more results
if (result .eqs. "") break
# No wildcard specified
if (result .eqs. old_result) break
old_result = result
filename = f$parse (result,,, "NAME")
filetype = f$parse (result,,, "TYPE")
file = filename + filetype
# Ignore directories
if (filetype .eqs. ".DIR") next
# Skip the ignored file types.
skip = "," + (filetype - ".")
if (debugging) wse "skip: ", skip, " ignoredtypes: ", ignoredtypes
if (f$locate (skip, ignoredtypes) .ne. ignoredtypes_len) {
if (debugging) wse "Skipping ", file
next
}
tcsfile = "[.TCS]" + file + "-TCS"
if (f$search (tcsfile) .eqs. "")
wso file, " does not have a ", tcsfile
}
exit
usage:
copy sys$input sys$output
//usage:
//tcs [ help | -h | -help | -? ]
// Output this help message.
//
//tcs admin FILE
//
// Initialize TCS file [.TCS]FILE from FILE.
//
//tcs get [-oOUTFILE] [-h] [-rM.N] FILE
//
// Get a revision of FILE from TCS file [.TCS]FILE.
//
// -oOUTFILE writes the output to OUTFILE.
// If -o is not specifed the output goes to standard output.
//
// -h Output history information instead of a revision.
//
// -rM.N Output revision M.N.
//
//tcs delta FILE
//
// Add a new version to TCS file [.TCS]FILE from FILE.
//
//tcs outdated
//
// List all the source files that are newer than their corresponding
// TCS file in [.TCS]
//
//tcs uncovered [FILESPEC] [IGNOREDTYPES]
//
// List all the files in the current directory that do not have
// corresponding files in [.TCS].
//
// If FILEPSEC is specified it should be a VMS wildcard that
// will be applied to the current directory for matching.
//
// If IGNOREDTYPES is specified it should be a single file type
// (without the ".") or a comma separated list of file types
// to be ignored.
//
// If one needs to specified IGNOREDTYPES but not FILESPEC just
// specify an empty string for FILESPEC.
exit 2 .or. QUIET
I decided to leave my tcsdiff.sdcl program separate for the
time being, since it was easy to convert to use with TCS files
in [.TCS].
I found it amusing when I recently was working on some
ratfor code soon after revising my SDCL
(Structured DCL preprocessor) and I realized that the reason that the
internal goto labels that I generate in SDCL start at 23000 because
the original ratfor command started at 23000, which was
carried over to the SWTOOLS VOS ratfor and the original
sdcl written by Sohail Aslam of the University of Colorado
at Colorado Springs, and from there to my version of SDCL.
The SWTOOLS VOS provides three shells, sh, hsh
(which adds a history mechanism), and esh (which adds
ed-style editing of the history). The release notes state
that the shells need the privileges DETACH and CMEXEC, but
under VMS 5.5-2 it turns out that they need the GRPNAM privilege
as well.
Well, it's my first program using SWTOOLS VOS this time around.
So, I finished by first program using SWTOOLS. It is a simple
directory lister, called l, that interprets all its arguments as as
file specifications to search for using normal RMS wildcards, or used
*.* if not arguments are specified. It only returns the files with
the highest version number, does not include the device, directory, or
version number in the output, and converts the filenames to lower
case. It should only be used to list files in the current directory.
I needed it because I do things like
l*.sdcl*.for|sedit's/{?+}$/ar uv src.ar $1'
and I don't want any version numbers to end up in the names of the
archive members, and so on.
Rafor is much better than Fortran 77. And Fortran has even more
footguns than C when it comes to using separately compiled functions
and subroutines, with no notices that I'd passed things with the wrong
type until I actually ran the program and got a register dump because
of an access violation.
I was not able to include things from
SYS$LIBRARY:FORSYSDEF.TLB using the Fortran INCLUDE
statement in ratfor, since it has its own include keyword , so I
separated that part of the program into its own Fortran source file.
Because that file wasn't run through ratfor I couldn't use the
standard macro definitions, like STDERR, so when I wanted to output
the filesnames, I just wrote a ratfor subroutine that would print them
and called that from the Fortran subroutine. A little clunky, but it
worked.
Annoying, the fc command (which compiles *.f files to
objects) doesn't work when run under mmk, so I had to write
a command procedure dommk that does mmk/noact/out=tmp.com
and @tmp.com to do the build. (I think this was somehow due to
having multiple levels of subprocesses: mmk's and
rc's.)
Anyway, the program ended up 48 lines of ratfor in three files and 84
lines of Fortran. The C program I started with was 170 lines (in one
file), but doesn't work with the SWTOOLS shell's notions of I/O
redirection.
Today I fixed a 41 year old bug in the SWTOOLS VOS sedit commmand!
At work part of my job is to maintain an emulated VAX/VMS 5.5-2 system
for one of our clients. I have been using the SWTOOLS Virtual
Operating System recently (Unix style
I/O redirection! And pipes!), and whenever I tried to use the sedit
command it would die with an integer overflow:
I use sed frequently on Unix, and it would be nice to have
the SWTOOLS equivalent on VAX/VMS. What could I do but set the
Wayback Machine
for 1986 and dive in!
The SWTOOLS VOS was written in Ratfor, which added structured
control structures and other niceties such as macros, include files,
and string constants to pre-1977 Fortran. I suspect that even as late
as 1986, when the version of the SWTOOLS VOS was released, lots of
installations were on machines that only *had* pre-1977 Fortran
compilers.
Here's an example of Ratfor source, from sedit.r:
# copyf - copy file name to opened file fdosubroutine copyf(name,fdo)integer name(ARB)integer fdointeger fdi,iinteger opencharacter c,namstr(MAXLINE)character getchfor(i=1;name(i)!=EOS; i=i+1) #get name into stringnamstr(i)=name(i)namstr(i)=EOSfdi=open(namstr,READ)if(fdi==ERR)call cant(namstr)while(getch(c,fdi)!= EOF)call putch(c,fdo)call close(fdi)returnend
Notice that the character variables got changed to LOGICAL*1?
That turns out to be 1 byte variable...
So, to start, I needed to run sedit in the debugger. That's easy
enough:
% cd ~tmp
% mkdir sedit
% cd sedit
/dua1/software/swtools/tmp/sedit/
% get -r1.5 ~src/sedit.tcs >sedit.w
% ar xv sedit.w
patdef
csedit
sedit.r
sedit.fmt
% rc -d -v sedit.r
~bin/ratp1.exe sedit.r | ~bin/ratp2.exe >sedit.f
~bin/fc.exe -v -d sedit.f
for/noop/object=sedit.obj/nolist/check=all/debug=all sedit.f
~bin/ld.exe -v -d sedit.obj
$ link/exe=dua1:[software.swtools.tmp.sedit]sedit.exe/debug/nomap sys$input/opti
ons
dua1:[software.swtools.tmp.sedit]sedit.obj,-
dua1:[software.swtools.bin]rlib.olb/incl=(tools$main),-
dua1:[software.swtools.bin]rlib.olb/libr
% echo Hello | sedit "s/H/J/"
%DEBUG-I-CANTCREATEMAIN, could not create the debugger subprocess
%DEBUG-I-CANTCREATEMAIN, could not create the debugger subprocess
-LIB-F-NOCLI, no CLI present to perform function
-LIB-F-NOCLI, no CLI present to perform function
%DEBUG-I-SHRPRC, debugger will share user process
%DEBUG-I-SHRPRC, debugger will share user process
VAX DEBUG Version V5.5-023
%DEBUG-I-INITIAL, language is MACRO, module set to TOOLS$MAIN
DBG> go
%DEBUG-I-DYNMODSET, setting module DOCOM
%SYSTEM-F-INTOVF, arithmetic trap, integer overflow at PC=0002E345, PSL=03C00022
break on unhandled exception preceding DOCOM\%LINE 20
20: IF (.NOT.(CMD .EQ. 97))GOTO 23162
DBG> go
%DEBUG-I-DYNMODSET, setting module DOCOM
%SYSTEM-F-INTOVF, arithmetic trap, integer overflow at PC=0002E345, PSL=03C00022
break on unhandled exception preceding DOCOM\%LINE 20
20: IF (.NOT.(CMD .EQ. 97))GOTO 23162
DBG>
Ok, line 19 is where the the integer overflow happened. BUF is an
array of INTEGERs, while CMD is a LOGICAL*1, which, if I
look at the sedit.r source:
# docom - execute a single command at buf(i) on linbuf and linenosubroutine docom(i,linbuf,lineno)character linbuf(MAXLINE)integer i,linenocharacter cmdinteger k1,k2include cseditcmd=buf(i+COMMAND)…
started as a character and got translated to a LOGICAL*1, the
one byte logical data type. Normally that's ok, because we're just working
with ASCII, so any character should fit in in one byte. Lets take a look:
DBG> ex buf[i+5]
DOCOM\BUF[19]: 00000173
DBG>
Oh dear. That's hexadecimal 173, which is decimal 371. No wonder we
got an arithmetic trap, integer overflow.
If I look for occurrences of buf in sedit.r (using emacs
18.55.105) I see lots of lines like where buf is passed to
subroutine addint, which is not documented in the manual pages,
unfortunately, but is in the runtime library RLIB.OLB. On
VAX/VMS the source for that is in ~vms/rlib.w:
# AddInt Put int into intara if it fits, increment j# works with an array of integersinteger function addint(int,intara,j,maxsiz)integer int,j,maxsiz,intara(maxsiz)if(j>maxsiz)return(NO)intara(j)=intj=j+1return(YES)end
Yes, that's just putting an integer in an array and incrementing the
variable that stores where the next one goes.
Some of the things they're adding to buf are clearly integers
(nlines, and probably NOTSTATE and OKSTATE), but those
EOS are the SWTOOLS VOS equivalent of the C '\0' that ends a
string, so those particular ones definitely hold characters.
Well, if I want to debug that I'm going to have to rebuild the SWTOOLS
VOS with debugging turned on. Investigating it turns out that I need
to add /DEBUG to the macro and fortran
commands in asmlib.com, asmprm.com, build.com,
and compile.com in ~bin, and then execute
~bin:toolgen.com.
Once that is done I start stepping through various routines.
Eventually I figure out that it buf(19) is the element that ends
up with the 173 hex, and set a watch on it. And indeed it is getting
passed a 173. Sigh.
Doing a SHOW STACK in the VMS Debugger shows that the
addint() call that dies is in the subroutine compil(). Lets
take a look:
DBG> type compil\50:65
module COMPIL
50: STATUS = ADDINT(LINE2, BUF, LASTBF, 20000)
51: COM = CLOWER(LIN(I))
52: IF (.NOT.(COM .EQ. 33))GOTO 23068
53: STATUS = ADDINT(1, BUF, LASTBF, 20000)
54: I = I+1
55: CALL SKIPBL(LIN, I)
56: COM = CLOWER(LIN(I))
57: GOTO 23069
58: 23068 CONTINUE
59: STATUS = ADDINT(0, BUF, LASTBF, 20000)
60: 23069 CONTINUE
61: STATUS = ADDINT(COM, BUF, LASTBF, 20000)
62: IF (.NOT.(COM .EQ. 97 .AND. FD .NE. 0))GOTO 23070
63: STATUS = ADDINT(0, BUF, LASTBF, 20000)
64: STATUS = DOTEXT(FD)
65: GOTO 23071
DBG>
Line 61 is the one that dies. Here addint() is being passed the
variable COM. Lets take a look at the Ratfor source for
its definition:
# compil - "compile" command in lin(i) from file fd, increment isubroutine compil(lin,fd)character lin(MAXLINE),com
Oh, look, com is a character, that gets translated by
Ratfor to a LOGICAL*1, and it is a byte, while the integers
that get passed in are longwords, which are 32 bytes long… Uh,
doesn't Fortran pass everything by reference? So addint() gets
passed the address of a byte, but interprets it as the address of a
longword, so it picks up 3 extra bytes. For whatever reason, there is
a 1 in the byte right next to the byte for com, so we get (hex,
remember) 173 instead of 73, which is the ASCII for s, which the s
in our sedit "s/H/J/"!
Ok, that should be easy to fix: introduce an integer temporary
variable, assign com to it, and pass it instead.
Here's the VMS diff:
************
File DUA1:[SOFTWARE.SWTOOLS.TMP]SEDIT.R;6
206 integer comint
207 integer fd
******
File DUA1:[SOFTWARE.SWTOOLS.TMP]SEDIT.R;1
206 integer fd
************
************
File DUA1:[SOFTWARE.SWTOOLS.TMP]SEDIT.R;6
237 comint = com
238 status = addint(comint, buf, lastbf, MAXBUF)
239 if (com == APPENDCOM & fd != NO) {
******
File DUA1:[SOFTWARE.SWTOOLS.TMP]SEDIT.R;1
236 status = addint(com, buf, lastbf, MAXBUF)
237 if (com == APPENDCOM & fd != NO) {
************
Number of difference sections found: 2
Number of difference records found: 3
DIFFERENCES /IGNORE=()/MERGED=1/OUTPUT=DUA1:[SOFTWARE.SWTOOLS.TMP]SEDIT.DIFF;1-
DUA1:[SOFTWARE.SWTOOLS.TMP]SEDIT.R;6-
DUA1:[SOFTWARE.SWTOOLS.TMP]SEDIT.R;1
One of the things I've always found interesting about the LBL Software
Tools VOS was how its source files were structured. I'll include here
page 10 of the release notes from the release on the DECUS LT86 tape:
Release Notes
Source File Structure
The source code for `tool' is contained in a file [...SRC]tool.tcs
(if the tool is portable across operating systems) or
[...VMS]tool.tcs (if it is an VMS-specific tool). This TCS source
file contains an edit history of all changes made to the source. The
output of the `get' utility operating on a `.tcs' file results in a
file (tool.w) which is all of the environment necessary to rebuild
the tool, provided that the VOS is operational. The tool.w file is
an archive containing:
1. All of the files "included" by the ratfor source code.
2. The ratfor source file, tool.r.
3. The format input for the manual entry, tool.fmt.
4. And optionally, any extra definition files needed to build
alternate versions of the tool (eg. sh => hsh).
As an example, suppose that you wish to change the subroutine
"module" in "tool". The suggested scenario is as follows:
$ !Fetch the file tool.tcs from the appropriate directory in the container
$ !file on tape into st_src
$ hsh
% get ~src/tool.tcs tool.w
% ar xv tool.w
% ar xv tool.r module
% ed module
(make changes and write file)
% ar uv tool.r module
% rc -v tool.r
% (test out new tool. repeat last three steps until satisfied.)
% ed tool.fmt
(modify writeup to reflect changes)
% ar uv tool.w tool.r tool.fmt
% cp tool.exe ~usr/tool.exe
% delta tool.w ~src/tool.tcs
(Identify in the comments the reason for the changes,
and which modules changed.)
% format tool.fmt >tool
% ar uv ~man/s1 tool
% asam <~man/s1 | sort >~man/i1
Placing tool.exe in ~usr causes the shell to find your modified
version of "tool" rather than the distributed one. The last two
commands above cause the manual entry for `tool' to correctly
correspond to the utility itself.
-10-
(It should be % get ~src/tool.tcs >tool.w with a “>”
before tool.w in the above procedure. Otherwise the contents
are echoed to the terminal instead of being written to tool.w.)
So, all the source files (source code and documentation) are stored
in an archive file, which contains another archive file that has just
the ratfor source, from which you extract the modules you want to work
on, and at the end update the archives in the reverse order! This
made organizing all the source much simpler, especially considering
some of the porting targets for this software didn't have hierarchical
directory structures, only flat directories. So when moving around
parts of the system, like when you were preparing the distribution,
you normally only dealt with the top level archives, the .TCS
files, and only worked directly with the files that were contained
within that archive and its child archives if you had to make a change
to the source files. Very clever!
Lacking Natural Simplicity is one, not particularly flattering,
definition of sophisticated.
This blog chronicles my journey through our at times too complicated
and sophisticated world.