Jan 2014
Issue
Collins Software Newsletter
February 2014
March 2014
Issue

LIST OF: FINISHED

A tool to do anything you could ever want to do with data files...

Last month I introduced "List", a web page to demonstrate my concept of software robots. I have since expanded the page far beyond my expectations.

This newsletter concentrates on the "List of" web page.

LIST OF page


Client Side Processing: LIST OF performs all operations on the client hardware.

Some of the processing includes:

  • File translation to/from CG2, CSV, TSV, Fixed, Json, XML, SQL, HMTL
  • 2D / 3D Graphics
  • Sort on multiple columns
  • Filters
  • Find / Replace
  • PDF generation
  • Select Columns

I provide some 60 data files that can be loaded from my server. You can also load files from your disk, or simply edit a new file to process.  I also provide a way to load a web page from anywhere and extract links, text and tags as another source for tabular data to process.

The current version performs all operations in memory, (soon to be changed to remove this limitations).

Processing on the client goes along with my philosophy to perform only shared data access and security on the server, everything else on the client where possible. "SPEED IS GOOD..."


Other Uses: My use of the utility is for edits I commonly make to source code and other text files.  I constantly need to change the case of a list of values, moving data into/out of json, sql; creating html from a list of values.

My main resource for this has generally been manual or I write something on a case-by-case basis. I have used the "List of" utility a few times to create parts of the program, such as getting a list of the Unicode of a single quote. WordPad converts these values to a single quote and corrupts the CSV by adding unwanted quotes. 

Data Files: translating data between different formats has always been a major part of software development. I have some 25 years experience of providing database translation services. The "List Of" tool performs most of the capabilities needed in the processing of data files.

General Capabilities:

  • Select / Reorder Columns
  • Sort Columns
  • Change Case (upper / lower/ proper)
  • Quick Filters (equals, starts with, contains...)
  • JavaScript Filters, (anything)
  • Randomize Rows
  • Remove Duplicates
  • Limit number of rows

Fixed Record formats:

  • menu to select / change column spacing
  • ruler to describe column spacing

Translate-to formats:

  • comma separated values (csv)
  • fixed formats (blank or tab filled)
  • sql (t-sql, pl-sql, mySql, ms-access)
  • json
  • xml
  • html

Translate-from formats:

  • cg2 (my format)
  • csv
  • fixed text
  • html

I hope you will try using this utility for your translation needs.


HTML to tabular Data: I have included in the "List of" the ability to parse any html web page and extract Links, Tags, Attributes and Text.

The listing of web pages as pure text provides a unique view of online content:

  • Links, list all A tag links, the innerText and the title
  • Text, list all innerText of every tag or of selected tags
  • Attributes, a list of all attributes used by every element on a page
  • Tags, a list of all tag names used on a web page and the number of times it was used
Software Robots: a self-contained logic module to perform a set of actions from commands that humans can remember.

The "List of" web page is a demonstration of my concept of Software Robots.  The Software robots that were used in this example are:

  1. List  - get list files
  2. Convert - convert anything to anything
  3. Sort - multiple columns
  4. Draw - draw 2D / 3D cg2 files
  5. Filter - popup menu, edit JavaScript filters
  6. Fixed - popup menu, edit fixed columns tabs
  7. Find - popup menu, find/replace/go to line
  8. Dialog - popup menu, Edit Text
  9. Wait - working progress bar
  10. Cookie - save/restore client cookies
  11. Pdf - generate PDF document

The concept here is to have self contained modules that can be used anywhere. I will probably extend this concept to the "List of" web page so that it becomes a part of the "Convert" software robot. The Convert robot accepts commands such as "convert cg2 to csv" which takes a string/file and returns a string/file. To accept a command like "convert cg2 to ?" the robot could popup the "List of" form with the cg2 file loaded ready for the user to perform some interactive processing prior to the completion of the command.

I am moving more toward having the software robot do whatever is needed to complete the task asked of it. When this includes user interaction, then we will get to the point that software will have no limits.

The speed of software development will increase exponentially as I build more robots.


 

Do Not Reuse Code:

The building of the "List of" web page has reinforced my belief that code should never of reused. I have some 80,000 lines of code used in the application. I had one (1) line of contamination (code reuse) that has caused a failure by being out of context. This one line will take me a day or so to resolve properly. 

Author: Clif Collins
Houston, Texas

Feb 10, 2014
http://CollinsSoftware.com

email: c2@CollinsSoftware.com