eed - extensible editor

Why are there more than a thousand text editors out there?

Why is it that I, and so many others are so deeply dissatisfied with all existing text editors?

The most capable text editors like vi and emacs have a steep learning curve because their weirdly idiosyncratic interfaces were developed long before interface conventions eased the burden on users. Many of the most functional editors also tend to be enormous. I was shocked, on downloading emacs recently, to find that it is around 30MB! And that's without frivolous extras. (That's about a third the size of the entire Puppy Linux operating system including its text editors, wordprocessor, spreadsheet, video player, internet tools, and much more!) These high-level editors generally require learning specialised macro languages too (emacs, for instance uses lisp).

At the other extreme we have tiny, minimal text editors like leafpad, mp and others. Unfortunately, the smaller they are, the less useful. At the lowest end they have only the barest functionality, with no easy way to add more.

Many existing text editors, whether they are giant, complex ones, or small, simplified ones, are pretty inflexible. If they don't have some kind of macro language, then extending them means either spending a lot of time delving deep into the source code to work out how they were created, or waiting for the next upgrade. Also, no matter how big and complex the program is, there will always be some more features missing, resulting in code bloat and feature-creep.

What seems to be missing more than anything else is re-use. You can't re-use the interface methods that you learned for all other programs, and you can't re-use existing programs within a text editor, or re-use the text editor from within other commands. People have somehow become accustomed to the idea that a text editor neccessarily stands outside everything else. But why should it?

So here is how to build the perfect text editor. Apply the old Unix philosophy: build a complex thing out of many small parts, each of which does just one job, but does it very well.

I've been building a series of small RoxApps that work under Linux's Rox file manager. (If you use a different file manager they might not work.) They sit on my desktop as a lot of small icons to give me many useful text editing functions. The neat thing is they can be used with any text editor, and even in dialog boxes, online chats, wikis, and blogs. You can download them here. (Please note that this isn't complete. It might never be. In a way, that's kinda the point.)


insert

edit

sort

spell

s_r

wc

date

wrap

custom

br

case

head

tt

b

i

centr

blkq
Here are some of my most commonly used text editing RoxApps.

When I pause the mouse over them a descriptive tooltip reminds me of what the icon does. Here is the tooltip for the "case" icon.


The tooltip for the "case" icon.

I've added menus to many of them so that they actually make more commands available. For example, here is the menu that displays when I right-click the "case" icon. Compare it with the tooltip above.


The menu for the "case" icon.

A RoxApp is actually a special kind of directory which acts as if it is a program. Clicking on its icon (often given by a hidden image file '.DirIcon' inside the directory) runs a shell script, named AppRun inside the directory (it doesn't have to be a shell script; it can be any kind of program). Many of my examples use the simple languages sed and awk that are available to all Linux distributions. (A number of my scripts require sed version 4.2.2, or more recent, because it introduces the -z option which views strings as terminated by a zero value byte, allowing newline characters to be seen and operated on as any other characters.) My examples usually get selected text into the script via the shell command xclip -o then put the result back into the clipboard using xclip -selection c, which allows it to be pasted back with xdotool key ctrl+v to replace the original selection. For example, here is the code for the menu item highlighted by the mouse pointer in the menu screengrab above:

"up1st")
	# upper first letter each word, else lower
	xclip -o | sed 's/[[:alpha:]]*/\L\u&/g' | xclip  -selection c
	xdotool key ctrl+v
	;;

That's in the AppRun script which lays out all the choices to be selected from that icon's menu. The actual menu is set by the AppInfo.xml file (it also sets the tooltip).

All this is extremely easy to program and can be changed dynamically. For instance if I edit the tooltip in the AppInfo.xml of one of the icons then the changes are displayed as soon as they're saved. (I've included an example RoxApp in the download which explains all this in greater detail. You can look inside a RoxApp by either holding the shift key down when you click it, or else by right-clicking it and choosing "App dir [name]" > "Look inside".)

As a group of icons on my desktop, I use eed with several different ordinary text editors. I also commonly use it in file-selection dialog boxes when saving a file, or when renaming files. Another place it comes in handy is when writing a comment or filling out a form on a website. Several of the commands are even useful in the terminal. The eed programs aren't restricted to being used any particular place; they form a general-purpose editing suite that can be used almost anywhere you type in text.

If many small commands are used in this way, then when an additional command is wanted, it is simple to add it. If some are not needed then they're not loaded. The flexibility gained is extraordinary. We can use the eed commands within other scripts and use other scripts inside eed commands. Such an editor could even be used as a filter for other programs, configuring it differently for each purpose.

Many of my eed functions require xdotool and xclip and sed version 4.2.2. or greater, so you must install them for most of the functions to work. Most Linux distributions have access to those programs and may already have them installed. If not, just use your distribution's program installer, or else search on the net for them and compile and install them yourself.

If sed version 4.2.2 or later isn't available directly though your updating tool you can find the latest version of it at http://ftp.gnu.org/gnu/sed/.
The xclip tool is most likely already installed or easy to install, otherwise you can get it from https://sourceforge.net/projects/xclip/.
Control of the GUI from within scripts is made possible by xdotool which, if it isn't available through your software installer, can be downloaded from https://github.com/jordansissel/xdotool.


Unix and Linux have, by my count, around 55 text editing commands already. Instead of duplicating their functions, why not use them and simply add more?

basic text commands
grep search a file for a pattern
sort sort lines of text files
uniq report or omit repeated lines
tsort topological sort
head output the first part of files
tail output the last part of files
tailf dynamically follow the growth of a log file
column columnate lists
line read one line (always prints a newline at the end)
rev reverse the character order in every line of a file
ul do underlining on a terminal
cut remove sections (columns) from each line of files
join output for each pair of input lines with identical join fields
paste merge lines of files (line1_file1\t line1_file2\n line2_file1\t line2_file2...)
split split a file into pieces
csplit split a file into sections determined by context lines
nl number lines of files
ptx produce a permuted index of file contents
hexdump ascii, decimal, hexadecimal, octal dump
od dump files in octal and other formats

analytic commands
wc print newline, word, and byte counts for each file
aspell interactive spell checker (replacement for ispell)
ispell interactive spell checker
style analyses readability (length of words, sentences and paragraphs)
diction find doubled words and wordy or commonly misused phrases in sentences

primitive format and display for screen or paper
fold wrap each input line to fit in specified width
pr convert text files for printing (paginate or columnate)
fmt simple optimal text formatter
cat concatenate files and print on the standard output
tac concatenate and print files in reverse (last line first)
less display and navigate a text file (also search and bookmark lines)
lesskey lets you set the key commands that less uses
lessfilepreprocess input for less (open tar, gz, doc, bz, deb, rpm, pdf, etc) to /tmp
lesspipelike lessfile but sends to standard output
more browse a file a screenful at a time (primitive version of less)
pg browse pagewise through text files
col filter reverse line feeds from input
colcrt filter nroff output for CRT previewing

verify saved files
cksum write file cyclic redundancy checksums (CRC) and sizes
sum checksum and count the blocks in a file
md5sum compute and check MD5 message digest
sha1sum compute and check SHA1 message digest

converters
tr Translate, squeeze, and/or delete characters
recode converts files between character sets
iconv codeset conversion
unrtf converts RTF documents to html (default), plain text, ANSI, LaTeX, rtf
expand convert tabs to spaces
unexpandconvert spaces to tabs
tidy validate, correct, and pretty-print HTML files

compare
cmp compare two files
diff find differences between two files
comm compare two sorted files line by line

clipboard
xclip X-based commandline clipboard utility
autocutselsynchronise all clipboards
xcutsel
glipper GUI clipboard history manager
klipper
anamnesissimple python- and sqlite-based commandline clipboard history
xclipboard
xsel
xcut


As well as the above discrete commands Linux usually comes with 2 text-oriented scripting languages (awk and sed) and often with python and perl, which are full-blown programming languages. Scripts written in these four languages make it relatively easy to add commands to eed.

text-oriented scripting languages:
awk pattern scanning and processing language
sed stream editor (stream-oriented text editor)
python scripting language
perl scripting language


The nosql project did a similar thing for databases. Instead of requiring a large, monolithic database program, nosql uses existing commands in Linux, plus a number of special-purpose ones written in C, awk, shell-script, or perl. Some of these would be useful for eed. They illustrate the simplicity of extending such a concept.

nosql tools:
addcolumn (awk) Create new empty columns of a table at right
addrow (awk) Append a new empty record to a table
awktable (c) Wrapper for 'awk' program to process a table
bsearch (c) Search a table for lines matching a specified key
column ---> getcolumn
compute (c) Compute an arbitrary expression using column names
constraint (awk) database schema and referential-integrity processor
csvtotable (c) Convert a CSV file into a table
ctime (sh) Turn column containing seconds since epoch into local time
depend ---> deptable
deptable (sh) Test for functional dependency betwen two columns
edittable (sh) Use an editor to edit or modify a table
envtotable (awk) Convert ENVIRONMENT into a one row table
filemode (c) Output octal file permissions of files named
filter ---> filtertable
filtertable (c) Run standard utilities against a table
formtable (perl) Print an arbitrary style report of a NoSQL table
frametable (sh) Beautifier for easier table editing
getcolumn (c) Pick columns by name, output columns in listed order
getrow (c) Select rows based on arbitrary AWK expressions
gregorian (awk) Translate selected date columns from Julian to calendar
index ---> indextable
indextable (perl) Generate table index files to be used by 'search'
islist (awk) Check whether an input file has a valid 'list' format
istable (awk) Check whether an input file has a valid table format.
itable (perl) Interactive front-end to NoSQL
jointable (c) Join of two tables on a common field
julian (awk) Translate selected date columns from calendar to Julian format
justify ---> prtable
keysearch (c) Fast search for rows that begin with a given string
ldaptolist (sh) Queries an OpenLDAP server and formats a NoSQL list
listtotable (awk) Take a file in 'list' format and makes it into a table
makeschema (sh) Build a database schema from a list of tables
maketable (awk) Build a valid table header from an xref file
muxtosql (awk) Unordered sequence of name/value pairs to SQL statements
muxtotable (awk) Unordered sequence of name/value pairs to NoSQL table
mysqltotable(sh) MySQL server query to NoSQL table
nblparser (awk) Experimental NoSQL Brokering Language (NBL) interpreter
nltable (awk) Inserts a unique record identifier into a table
nosql (sh) Main wrapper for all NoSQL commands
notcolumn ---> getcolumn to remove one or more columns
number ---> nltable
project ---> getcolumn
prtable (sh) Table formatter for character displays
psqltotable (sh) PostgreSQL server query to NoSQL table
random ---> rndtable
rdbtotable (awk) Convert an /rdb or RDB table into NoSQL format
rename ---> renamecol
renamecol (awk) Rename a column
repair ---> repairtable
repairtable (awk) Append empty data fields to rows to match table header
rmcolumn ---> getcolumn to remove one or more columns
rndtable (awk) Pick one or more table records at random
row ---> getrow
search ---> searchtable
searchtable (perl) Select rows on a multi-column key of a sorted/indexed table
seektable (c) Extract rows at selected offsets of indexed NoSQL table
select ---> getrow
setenv (sh) Set up the proper shell envoronment for NoSQL (if eval'd)
setnames (awk) Set new column names on the input table
slapd-bind (sh) Experimental OpenLDAP 'back_shell' BIND processor
slapd-search(sh) Experimental OpenLDAP 'back_shell' SEARCH processor
slapdtonbl (awk) Experimental OpenLDAP 'SEARCH' to NBL converter
sorttable (c) Sort a table by one or more columns
soundex (awk) Compute Knuth's soundex codes for selected columns
sqlitetotable(sh) Build a valid NoSQL table from a SQLite database table
subtotal (perl) Output subtotals of a table
summtable (perl) Show statistics about a NoSQL table
tablecat (awk) Concatenate multiple compatible tables
tabletocsv (awk) Convert a table into CSV format
tabletolist (awk) Convert a table into the corresponding 'list' format
tabletordb (awk) Convert a table into /rdb format
template ---> xreftable
tmptable (sh) NoSQL temporary table creator for pipelines
total ---> totaltable
totaltable (awk) Compute table totals
union ---> uniontable
uniontable (sh) Build the "Universal Table" of one or more tables
unique (awk) Make a table unique on the primary key field
unsorttable (c) Shuffle rows in a NoSQL table
update ---> updtable
updtable (awk) Insert/update/delete table rows using an edit table
userproc (sh) Applies local processing to query results
usleep (c) Command-line interface to usleep(3)
viewtable (perl) Visualize a table in a nice list-like format
widest (awk) print the max. width of each column in a table.
xreftable (awk) Build a table template file for the input table


Here are some additional programs and scripts of use.

extras
wf word frequency
unwrap remove newlines from inside paragraphs
gtkdialog3file open/close dialogs, GUIs for many low-level commands
cal outputs a calendar
html-tag-lowercasemakes all html tags lowercase
gres(Global Regular Expression Substitute) grep-like find and replace
searchreplace search-replace with GUI and regex.


I don't have these yet. The most important is the first one: the main interactive window.

wanted
scr Interactive screen display.
Should be able to add menus dynamically from a simple text config file.
Tabbed display would be useful.
Inbuilt toggling soft-wrap would simplify matters too.
Nice if it could display styles (fonts, style, color, size, align), making it wysiwyg html-capable.
find interactive find with GUI and regex.