LIST OF: FINISHED
A tool to do anything you could ever want to do with data files...
Last month I introduced "List", a web page to demonstrate my concept of
software robots. I have since expanded the page far beyond my
This newsletter concentrates on the "List of" web page.
LIST OF page
Client Side Processing: LIST OF performs
all operations on the client hardware.
Some of the processing includes:
- File translation to/from CG2, CSV, TSV, Fixed, Json, XML, SQL,
- 2D / 3D Graphics
- Sort on multiple columns
- Find / Replace
- PDF generation
- Select Columns
I provide some 60 data files that can be loaded from my server. You
can also load files from your disk, or simply edit a new file to
process. I also provide a way to load a web page from anywhere and
extract links, text and tags as another source for tabular data to
The current version performs all operations in memory, (soon to be
changed to remove this limitations).
Processing on the client goes along with my philosophy to perform
only shared data access and security on the server, everything else on
the client where possible. "SPEED IS GOOD..."
Other Uses: My use of the utility is for edits
I commonly make to source code and other text files. I constantly
need to change the case of a list of values, moving data into/out of
json, sql; creating html from a list of values.
My main resource for
this has generally been manual or I write something on a case-by-case
basis. I have used the "List of" utility a few times to create parts of
the program, such as getting a list of the Unicode of a single quote.
WordPad converts these values to a single quote and corrupts the CSV by
adding unwanted quotes.
Data Files: translating data between
different formats has always been a major part of software development.
I have some 25 years experience of providing database translation
services. The "List Of" tool performs most of the capabilities needed in
the processing of data files.
- Select / Reorder Columns
- Sort Columns
- Change Case (upper / lower/ proper)
- Quick Filters (equals, starts with, contains...)
- Randomize Rows
- Remove Duplicates
- Limit number of rows
Fixed Record formats:
- menu to select / change column spacing
- ruler to describe column spacing
- comma separated values (csv)
- fixed formats (blank or tab filled)
- sql (t-sql, pl-sql, mySql, ms-access)
- cg2 (my format)
- fixed text
I hope you will try using this utility for your translation needs.
HTML to tabular Data: I have included in the
"List of" the ability to parse any html web page and extract Links, Tags,
The listing of web pages as pure text provides
a unique view of online content:
- Links, list all A tag links, the innerText and the title
- Text, list all innerText of every tag or of selected tags
- Attributes, a list of all attributes used by every element on
- Tags, a list of all tag names used on a web page and the number
of times it was used
Software Robots: a self-contained logic module
to perform a set of actions from commands
that humans can remember.
The "List of" web page is a demonstration of my concept of Software Robots.
The Software robots that were used in this example are:
- List - get list files
- Convert - convert anything to anything
- Sort - multiple columns
- Draw - draw 2D / 3D cg2 files
- Fixed - popup menu, edit fixed columns tabs
- Find - popup menu, find/replace/go to line
- Dialog - popup menu, Edit Text
- Wait - working progress bar
- Cookie - save/restore client cookies
- Pdf - generate PDF document
The concept here is to have self contained modules that can be used
anywhere. I will probably extend this concept to the "List of" web page
so that it becomes a part of the "Convert" software robot. The Convert robot
accepts commands such as "convert cg2 to csv" which takes a string/file
and returns a string/file. To accept a command like "convert cg2 to ?"
the robot could popup the "List of" form with the cg2 file loaded ready
for the user to perform some interactive processing prior to the
completion of the command.
I am moving more toward having the software robot do whatever is
needed to complete the task asked of it. When this includes user
interaction, then we will get to the point that software will have
The speed of software development will increase exponentially as I build more
Do Not Reuse Code:
The building of the "List of" web page has reinforced my belief that
code should never of reused. I have some 80,000 lines of code used in
the application. I had one (1) line of contamination (code reuse) that
has caused a failure by being out of
context. This one line will take me a day or so to resolve properly.