Why I was a Korn Shell Fan
A lot of Linux fanboys these days love everything Gnu, and like to talk shit about non-Gnu stuff. Perhaps it’s just a modern-day version of the empty Commodore vs. Apple vs. TRS-80 arguments we had back in the day: they each had strengths and weaknesses, so a lot depended on what you wanted to do. And a lot had to do with what you were familiar with.
Nevertheless, there are reasons I prefer Korn shell. If you have different reasons, you may prefer other shells. And that’s just fine too—but if you’re just sticking with something because it’s all the rage, that’s not a good reason.
So let’s start with the small stuff and work our way up.
Floating point math
You don’t need it often, but when you do, you don’t
have to screw around using bc
or dc
to do
it.
Korn shell’s built-in sleep
accepts
floating-point values too.
print command
echo
has differing variants on different platforms.
Ksh’s print
works the same everywhere.
Compound variables
You can create a variable that has multiple fields in it. A
simple struct
of sorts. See typeset -C
,
which, by the way, you can do with…
Built-in documentation
Try it out: typeset --??
(or typeset
--man
) to learn about the myriad of options to typeset. No
longer do you need to pore through a crazy long man page to find
what you want, it’s right at your finger tips. Almost all the
built-ins have them. And the man page isn’t all:
-?
gives a short synopsis, and --help
shows the synopsis and option details.
And if you want to print that:
typeset --nroff 2>&1 | groff -man -T ps | ps2pdf - typeset.pdf
It also does --html
, --about
,
--short
, --long
. See getopts
--man
for more, and getopts --??help
for a full
list of display options.
Self-documenting getopts
Those man pages just mentioned? They come from the
getopts
string using Ksh’s long format. (The
commonplace short format is accepted too.) The long format is
described in the getopts
built-in man page, but you
may also need to cookbook from an example; I’ve included some
info after this essay. The format is a little weird, but easier
than writing raw man pages, which you can create with the
--nroff
option. And it’s used to provide the
same sort of short synopses, long synopses, and man pages on
demand. Lastly, the getopts
long options string is
both directive and documentation, ensuring docs and code are in
sync.
Good documentation
Bash has decent documentation, comparable to the quality of Korn
shell. zsh
documentation, however, can be sparse,
especially regarding some of the command options. That and the
frequently-corrupting shell history (especially if you do a lot of
multi-statement commands on the command line, which seem to
encourage the behavior, then are lost uninvoked when the corruption
happens) drove me away.
FPATH
You can set a function path. If a function is requested and not known, Korn shell searches and loads it if found. You can have a library of functions, without having to copy-and-paste your code (or the bugs that go with that code).
True local variables
In modern shells, there are two ways to declare functions:
name () { ...
and
function name { ...
Usually, these behave the same. And because of backward combatability, the former must behave like the old Bourne Shell, which was insanely simple by today’s standards, to the point of being broken by modern expectations. But it’s a long-lived problem in software: we can’t fix anything because of all the things the fixes would break.
One of those simplicities is the lack of local variables.
Recursion wasn’t possible in the old world. (Well, maybe with
judicious use of eval
. Is your hair gray yet?)
And looking back, Bourne shell only had the first form of function invocation. Korn shell introduced the second form with different behaviors to correct the defects.
So in Korn shell, if you create a local variable in a C-style function declaration (with the parenthesis), the shell ignores you and uses a global variable.
When you use function
, you do get local
variables. Real, true, lexical-scoped function scope locals: the
functions you call can’t see them, nor fudge them up. And if
they forget to declare their own locals, they pollute the global
namespace instead of silently screwing up someone else’s
locals.
In Bash, locals are created in the current context. But they are dynamically scoped, thus available to every function you call. Or any function they call in turn. And if one of those functions forgets to declare a local, it may very well unintentionally alter a variable belonging to another function. Pollution of the global namespace isn’t good, but it’s less hazardous than this nonsense.
Pipeline variable retention
When running a pipeline, Korn shell retains any values set at the end of the pipeline. So this construct works:
grep "^#" < source | sed -e ... | some other stuff | read variable
But in Bash, it doesn’t. The variable is read into a subshell which is disbanded when the pipeline completes. Now, the argument from Bash proponents is to just do this:
variable=$(grep "^#" < source | sed -e ... | some other stuff)
And that’s true enough when all you want is one line out of a file. But what if you want to stick a while loop on the end?
float total=0
grep "cleared" < checkbook | awk '{print $NF}' |
while read value
do
let "total += value"
done
print "The total is $total."
In Bash, this wouldn’t work because it’s using
floating point math. But even if it was integer math, it
wouldn’t work because it would total up the value, then the
pipeline would close and the subshell would exit, taking
total
with it.
I realize there’s other ways to do this particular
example. It wouldn’t be hard to run that through
sed
and make input you could feed to bc
or dc
to get a total. Or you could feed in a sentinel
value to cause the loop to print its value, and capture that. But
this is clearer and simpler. No hurdles.
And even if this is a hokey example, the concept stands. The pattern of read input, process it through a pipeline, and then amalgamate/ aggregate it at the end is not rare.
Note that since Bash 4.2, this is behavior is available via
shopt -s lastpipe
. However, this behavior only takes
effect if job control is disabled, so it won’t work on the
command line or in any functions you have loaded.
Interactive Use
For interactive use, the two shells seem very similar to me. Both have emacs and vi edit modes, quoting is identical (or nearly so) between the two, and many of the built-in variables are the same.
Bash has the bind
command for keymapping;
ksh
has a KEYBD trap. I’m not sure which is
better.
To the extent that Bash has emulated lots of Korn-shell pioneered behaviors, they’ve done well. Except the pipeline handling.
I have spotted at least one feature—case manipulation in parameter substitution—that’s been added to ksh based on its presence in Bash. So the copy-catting is going both ways.
And there are other unique Bash features, like the hook function
for missing commands; ksh
has no equivalent. (If I was
implementing it, though, I’d make it a trap. It would make
more sense than a magically named function.)
Imperfections
I do worry about the state of Korn shell. David Korn has
moved on, and though AT&T has open sourced the code, will Korn
shell be kept up? Only time will tell.
The code is old, and it looks it; like something written in
the 80s. How much edit fatigue does it have? I’ve delved
enough to know there code is ugly. And I’ve watched the
current maintainers, at least one of whom seems to be a bull in a
China shop. He’s done some awesome stuff—ripping out a
legacy, AT&T-proprietary memory manager and moving to standard
malloc
—but he also aggressively removes anything
he doesn’t care about, like Windows support. (I don’t
care about it either, but I’ll bet there’s someone who
does.)
Unfortunately
Unfortunately, it’s 2020, a year that in the future we’re going to wish we could just forget.
It’s also the year “ksh2020” started making the rounds. Several years ago, the code was open-sourced; after a few years of neglect, a few folks took notice and started maintaining the code base. I contributed a few minor fixes myself.
Unfortunately, one developer has proven to be a “bull in a pottery shop”, and while some of his efforts are things that definitely needed to be done— stripping out a proprietary memory manager and just using malloc, for one— he never wanted to hear different opinions.
He asked on ksh
developers discussion if anyone
used ksh
on Windows, and when nobody responded, he
ripped the Windows support out. I don’t think he asked
anywhere else, where there might have been Windows users to
object.
Someone had provided a patch to add a new format for shell
parameter expansion, to allow capitalization, but it was poorly
implemented, not entirely compatible and didn’t quite work
how you’d expect. But I thought it would be handy, avoiding
lots of upper=$(print -- "$thisvar" | tr 'a-z'
'A-Z')
sequences, if it worked. I developed a patch, but
there was some complaint (which I think was legitimate), so I
reworked my patch again, and came up with a very nice
implementation that was compatible with the bash
version but better. Now krader faught it because it wasn’t
traditional ksh
, and why were we trying to be like
bash
anyway? He shot down the fix and persuaded them
to rip out the feature entirely.
They did take my patch for fixing prompt column counting, adding
some additional escape sequence support. Later on I had some reason
to look at some older ksh
code, and stumbled across
the column-counting code—where I saw some code that
wasn’t mine, but did something similar. I pulled up the later
code and compared—parts had been diked out, and I unknowingly
replaced them because their removal had broken things. I never
tracked down who broke it, but I have my suspicions.
After a while, I realized my help wasn’t really wanted. In any discussion, Krader’s mind was set, his touted years of experience showing he was more right than anyone, and those who disagreed were wrong. Trying to help was more frustration than it was worth, so I stopped trying. I’m aware I’m not alone.
In July 2020, I upgraded to LinuxMint 20, which came with
ksh2020
for ksh
. I started having
weirdness with file globbing, where ls
would list
files but echo *
would return nothing; a single star
matched nothing even though there were a bunch of files and
subdirectories in the current directory.
If globbing cannot be trusted, the shell cannot be trusted.
And after ksh2020
flopped for being a buggy piece
of trash, he bogged off and left his broken handiwork rotting on
GitHub. Open-source developers should consider the lesson offered:
how one self-appointed, over-confident, mindless-of-others,
reckless developer who is blind to any damage he does can taint a
codebase and destroy others' inspiration to contribute.
There seem to be some saner folks trying to start again from an earlier, untainted version, but only time will tell.
Replacement: zsh
Interactively, I have settled on zsh
as a
replacement. It isn’t a drop-in replacement because there are
incompatibilities, but it’s somewhat close. There are various
quirks moving to zsh:
- To void frequent “shell history corrupted” issues,
don’t
export HISTFILE
; if you do and ksh gets invoked as a subshell, it appends to the history file and giveszsh
a concussion. - Disable
^D
handling:bindkey -r '^D'
- Disable escape-slash handing:
bindkey -r '^[/'
- Write a handler for ‘
v
’ in vi command mode to edit the command line invi
- Update
^L
handler, which should clear the screen and the scrollback buffer, thank you very much.
There were also scripts that required updates:
- I had to rewrite bits of my shell library (functions in my
$FPATH
) to accommodate variations, especially the lack of a-q
option towhence
. - Script variables named
$path
need to be renamed, aszsh
sets it to a space-separated version of$PATH
, a crappy unexpected behavior imported fromcsh
- Similarly,
$PROMPT
is linked to$prompt
. - Array indexing starts at 1, not at 0.
zsh
documentation is a hassle: there’s a lot
of it, it’s split across several man pages. Losing the
--help
and --man
options to the built-ins
is inconvenient.
But so far, it’s proving to be a viable transition. Some
things are improved, like the column-counting: there are escape
sequences that direct zsh
to stop and resume counting.
It means a little more work to set the prompt, but avoids an ugly,
unreliable counting algorithm hard-coded in the shell.
Conclusion
Admittedly, I’m more familiar with Korn shell, and
that’s an influence. While I can use Bash just fine
interactively, I find it frustrating to try to script with. After
ksh
, bash
seems just broken—and in
some cases, it’s probably because I’m not used to it.
But in other cases, Bash requires jumping through hoops that
aren’t necessary when working in Korn shell.
So, in programming features, ksh
seems far ahead to
me. Unfortunately, its maintenance and future are unclear.
zsh
shows promise as a sanctuary for Korn shell
users who are pushed away by a rogue developer and entropy.
I haven’t seen misbehavior from my existing
ksh
scripts, and unless I do, I’m letting that
sleeping dog lie. But should it be necessary, zsh
does
seem like a viable candidate to move to without completely
rewriting.
Despite name recognition, bash has a long way to catch up,
although with constant attention it could catch up. I get the
sense, though, bash
isn’t even trying; they
don’t mind, for example, that the shell can’t do
floating point math.
Korn Shell’s self-documenting getopts
Insanely old Korn shells or other shells portending to be Korn Shell might not support the advanced getopts. If you are worried about this, you can detect availability with this construct:
if [[ $(getopts '[-][12:abc]' flag --abc; print -- 0$flag) == "012" ]]
then
GETOPTS_STRING=$'
...new-style-getopts-document
here...'
else
GETOPTS_STRING="old-style-getopts-string here"
fi
The $''
quoting uses C-style quoting within, to
keep you sane.
In your parsing loop, provide the selected string:
while getopts "$GETOPTS_STRING" flag
do
case "$flag" in
... handle your options flags here
esac
done
So what do you put in the new-style getopts?
A section title and a paragraph for it in the resulting man page:
[+TITLE?example - how to use the \bksh\b(1) getopts string]
If you need a new paragraph, leave out the title:
[+?This is another paragraph under the last title.]
To insert a definition list instead of a paragraph, wrap the entries in braces:
[+EXIT STATUS]
{
[+0?The command succeeded.]
[+1?The file was not mangled correctly.]
[+2?The file could not be found.]
}
To define an option:
[s?Flag with short option only.]
[f:long-flag-name?Description of the flag.]
[p:option-with-argument-name?Description]:[argument-name]
[n:numeric-option-name?Explanation]#[argument-name]
[x:?This flag has no long-option name]
[12:flag-with-only-long-name?There is no short-form of this flag.]
The letter after the opening bracket is the short option. If a long option is used, it’s converted to the short form when provided to you. If you don’t want a short form, provide a 2 or more digit number to be used instead. Numeric argument specifications are enforced, but note 0xhex is allowed and decimal numbers (0.5) are an error.
For options with arguments, you can include a list of values.
These are rendered in documentation, but they are not used for
getopts
processing. So don’t expect any
restrictions they document to be applied by
getopts
.
[l:flag-with-list?Flag that takes only one of the defined values.]:[argument-name:type:attributes:=default-value]{
[+one?This is the \afirst\a choice.]
[+two?This is the \asecond\a choice.]
[+three?This is the \athird\a choice.]
}
To include information about the script’s creators:
[-title?Name and contact information]
For example, [-author?Perette Barella]
or
[-license?This is free software released under the MIT
license.]
Parameters go after a blank line following all your other text:
from-file to-file
If you have multiple forms, you can list them on separate lines:
from-file to-file
from-file ... to-directory
Lastly, there’s a handful of useful escape sequences: \b (backspace) toggles strong emphasis (bold). \a (bell) toggles emphasis (italic/underline). \v (vertical tab) toggles fixed-width display.
Putting it all together, here’s a demo program you can tinker with:
#!/bin/ksh
while getopts -a getopts_info $'
[+NAME?getopts info - document describing the ksh long-form getopts]
[+TITLE?This is a section and a paragraph in the resulting document.]
[+SOME OTHER SECTION?This is a spare section]
[+?Here is an example of a \adefinition list\a]
{
[+SOME] ?Text inside this \vdefinition list\v.]
[+OTHER]?And text for another \vdefinition list\v entry.]
}
[+?This is a continuation paragraph]
[s?Flag with short option only.]
[f:long-flag-name?Description of the flag.]
[p:option-with-argument-name?Description]:[argument-name]
[n:numeric-option-name?Explanation]#[argument-name]
[x:?This flag has no long-option name]
[12:flag-with-only-long-name?There is no short-form of this flag.]
[l:flag-with-list?Flag that takes only one of the defined values.]:[argument-name:type:attributes:=default-value]{
[+one?This is the \afirst\a choice.]
[+two?This is the \asecond\a choice.]
[+three?This is the \athird\a choice.]
}
[-Author?Perette Barella]
[-Copyright?This document is public domain.]
argument argument2 ...
Other forms of this command
[Note that at this point, this is plain text]' option "$@"
do
print -- "Argument: '$option' optarg:'$OPTARG'"
done