The following schematic shows the operating principle underlying LBX (ListBoxBrowseExtender) browses. The same principle applies whether the DATA SOURCE is local TABLE(s) via Clarion File Driver or remote TABLE(s) via CHT Client Server.
• Circle 1 represents the DATA SOURCE
• Circle 2 represents the LBX MAIN BROWSE QUEUE
• Circle 3 represents the LBX SHADOW BROWSE QUEUE
Yes, LBX uses two entirely separate browse queues underlying ONE Clarion LIST BOX control on your window. Clarion enables us to change the queue that a list box is looking at by simply issuing a PROP:QUEUE command to the list box stating the new queue name. Both queues may contain the same or different data sub-sets at the same time and something as simple as a button-push can change the data being viewed in the LIST BOX from one queue to the other and back.
Behind this dual queue design is the intention of MINIMIZING the interaction between Cirlces 1 and 2. The more times that fresh queries are sent direct to the the DATA SOURCE to refill circle #2 with fresh data, the more likely there's going to be speed bottlenecks and or slowdowns in the operation of filling the browse represented by circle #2.
These types of bottlenecks are most likely to happen in busy environments, with large-table data sets and over remote connections which inherently use a "thinner" or "slower" data connection.
Coralling and operating on data sets can be a lot like herding cats. So instead of coralling all 100,000 cats we corral just some of the cats (fat cats, skinny cats, lazy cats, young cats, old cats, etc. etc.) and work on various sub-sets of these individually. If, for instance, the fat-cats overall set contains both orange, gray and amber tabbies, we can load queue #1 (Circle #2) with a full set of fat-cats and the sort out the orange, gray and amber tabbies separately from Circle #3 all without needing to go back to the DATA SOURCE for a fat-cats sub-set.
The second LIST BOX QUEUE (circle #3) is always filled from LIST BOX QUEUE #2. Such that filling the circle #3 queue with either ALL the records in #2 or with SOME of the records in circle #2 requires no reference to the DATA SOURCE and, hence, no interaction with the back end to slow things down. Records simply travel in memory from circle #2 into circle #3 using a QUERY (i.e. a filter) that drills down and further refines the circle #2 data set.
The number of records cached in circle #2 does not change. Only the records matching a user-initiated query are COPIED into circle #3. Call this operation what you like, "sub-filtering" or "drilling-down" or "further-refining the data set".
Records can be edited or processed from circle #2 or from circle #3 without a whole lot of table (DATA SOURCE) interaction other than putting data changes back to the data source in real-time as they're being made.
Ideally, when a record is edited (or changed by a process) it does not have to travel both ways (i.e. to the Data Source and back from the Data Source). Since changes are made locally and transmitted TO or PUT TO the DATA SOURCE, we already know, and can reflect the changes directly in the DATA QUEUEs without needing to refresh them from the DATA SOURCE.
However, since changes are PUT to the DATA SOURCE in real-time, the original data source is always up to date with individual changes such that a refresh (new query) from #2 to #1 reflects all of the changes made to the data by all users as they work.
The less "Chatter" going on between circle #2 and circle #1, the more efficient the Data Design becomes. The more efficient it becomes the more simultaneous users it can handle.
More about this in an upcoming Part 4.
The first image below is a look at one of the two browse procedures from the Extensions template tree. I've highlighted the ListBoxBrowseExtender-related templates on the procedure.
The next images shows the compiled procedure running to indicate which of the templates above is which on the procedure interface.
If you've read any of our rants over the years you're aware that from an efficiency point of view the built-in behaviour of typical ABC browses to load records unfiltered into the browse on startup is just bad design, especially in a file-loaded listbox which is apt to load all the records in the table(s) underlying the browse, if you let it. Fine for small data sets, but a real speed-killer for large data sets.
Even with page-loaded ABC browses, a terrible burden is put on the back end (and on the data pipe) when loading the browse initially with ALL DATA IN THE FILE.
The purpose of this PreFilterButtons_LBX template is to give your users an option to appease their clarion-induced addiction of having the browse load SOMETHING on startup. That is, before the browse is told with a user query what records he/she ACTUALLY wants to see.
This template lets developers enter a generic filter like (NAME UNDER C) to load all the names starting with A and B, just so that some data appears when the browse opens up.
Of course, if your browse is of a VERY SPECIFIC function, then naturally, you might pre-filter the browse with a query like (ORDERS RANGE TODAY()-4,TODAY()) to see orders for the last 5 days including today.
Since LBX is a File-Loaded browse intended to work with BATCHES of data, it gives you at least a shot at loading a batch of data that may actually be of specific interest to the user starting the browse.
A Pre-Filter can also act as a background filter such that a second filter entered into the SortFieldFilter_LBX field creates a filter within a filter with the second filter acting inside the range of the pre-filter.
The two buttons in PreFilterButtons_LBX are as follows: The double-arrow button clears the pre-filter and resets the browse, while the single-arrow button re-establishes the pre-filter and resets the browse.
So lets call this Pre-filter by perhaps a more enlightening name, say, "BATCH FILTER", and here's why...
LBX is a file-loaded listbox and you want as much as possible to load a bunch of related records (related to what it is you're looking for or operating on) and then you want to sub-filter the BIG initial BATCH into smaller batches or into single records.
With LBX you can perform secondary sub-filter(s) (on the BATCH data instead of the WHOLE FILE) without impacting the FILE or having to go back to the file again and again. This works whether you're operating on LOCAL data or on REMOTE data.
LBX works with something we call a "SHADOW QUEUE". When you have a BATCH of data loaded into the listbox queue, you can filter it further (drill down into the batch), either with a built-in-locator (in filter mode) or with an on-screen filter like InQueueQueryControl_LBX. Once you execute this secondary query, the originally loaded BATCH of data is kept intact in the MAIN queue and the records matching the secondary query are loaded into the SHADOW QUEUE and appear in the listbox. When InQueueQueryControl_LBX is cleared the SHADOW QUEUE is switched out and the MAIN queue switched back in, still containing the original BATCH of records.
The upshot of this design is efficiency. It operates locally (in queue) on your data as much as possible without resetting either from the file or from the remote Client Server, until such time as you're done with the current BATCH and you want to switch in a new BATCH of records.
The template called SortOrderFieldFilter_LBX can be used to query for a new batch of records. Click on any column and then enter in data related to that column. No query string required, just some data to filter "in" records which match your data in a CONTAINS or STARTSWITH fashion. This particular filter does reset from the file so it can be used to reload a new BATCH of records having something in common that users are interested in seeing or operating on.
Reseting records from TABLES or CLIENT DATA SERVER:
• PreFilterButtons_LBX is a filter that filters on the way into the browse queue from the data file or data server.
• SortOrderFieldFilter_LBX is a filter that ALSO filters on the way into the browse queue from the data file or data server. This filter can be used in conjunction with the above Pre-Filter to provide the function of a dual filter or "compound" filter. To have SortOrderFilter_LBX operate all by itself without the PreFilter being in place, simply push the double arrow button of PreFilterButtons_LBX.Reseting records from the MAIN BROWSE QUEUE:
• Built-In-Locator can act as a locator or a filter using the MAIN QUEUE batch.
• InQueueQueryControl_LBX is a multi-column filter that also filters forward from the MAIN QUEUE batch. It appears to reset the browse, but it actually shows you a subset of the MAIN QUEUE BATCH in the "SHADOW QUEUE".
All of the above operates exactly the same way, whether you're displaying from local data tables located on the same machine as the browse procedure, or from REMOTE DATA tables located miles away via a CHT Client Server app.
In the HNDPEOPLE_LBX.APP we posted for you on Dec 15th, the two browses are operating strictly on LOCAL-TABLE data using one of the six data source options listed on the LBX template interface.
Here they are again...
It's clear from the labels on our two procedures that each of them uses one or the other of options 1 or 2.
More coming in Part 3.
This build was posted last night at 10PM DEC 15, 2016 (EST) and a number of you downloaded almost right away, between last night and approx 1PM today DEC 16, 2016 (EST).
We found a couple of issues this morning with that LBX implementation which required a small tweak and have corrected and re-posted. So if you downloaded/installed 20D.02.00 between the two times noted above, we suggest you run Webupdater C10 again to make sure you have the latest. Webupdater C10 knows whether anything has changed since you last updated so it won't download anything if you're up to date.
The demo app to consult is HNDPEOPLE_LBX.APP. We want to get rid of the prior demo app HNDPEOPLELBBX.APP so the name conforms to the correct acronym for ListBoxBrowseExtender, that is, "LBX" not "LBBX". For the time being we've simply copied HNDPEOPLE_LBX.APP over HNDPEOPLELBBX.APP and have supplied both apps so they are temporarily identical. But HNDPEOPELBBX.APP will disappear in the next update and be replaced solely by HNDPEOPLE_LBX.APP.
When you first open this application, even before you compile it, notice that it contains NO HAND EMBEDS. Not that we're against embedding, as you well know, but with LBX we want to provide the highest level of functionality possible from the template interface. More about the template in PART 2.
Here is a screen snap of the application sitting open on our desktop. Note that there are no little red icons beside the procedure names indicating hand-embeds.
BrowsePeopleLBXFillFromFile() (LBX #1), and
BrowsePeopleLBXFillFromView() (LBX #2)illustrate LBX browses.
Look closely when you run LBX #1 and note which fields are displayed. These are all fields from the PEOPLE file as suggested by the procedure name "BrowsePeopleLBXFromFile()". This browse loads and displays data from one file only.
Now look closely at LBX #2 and note which fields are displayed. This browse contains fields from PEOPLE as well as from PHONE, a related file. This browse is filled by looping a CLARION VIEW structure with ABCViewManager. Clarion VIEWS can traverse several tables at once assuming these tables are correctly joined in the dictionary.
So this browse displays fields from the PEOPLE table AND the PHONE table at the same time as suggested by the procedure name "BrowsePeopleLBXFillFromView()". This procedure loads and displays data from two files at once.
You're familiar with Clarion VIEWs, and how they work. We've written about this before, in this article available on our website:
Related-Table-Data On ABC BROWSES
We're re-focussing on this Clarion VIEWs concept because it's important to internalize what a VIEW does to the data that it reads. In short, a view lets you think of multiple joined tables as if they were ONE BIG FLAT FILE.
• CLARION VIEWS
Where CLARION handles the join using a CLARION VIEW and you treat multiple data tables via ABC ViewManager as if they were ONE Clarion Table using one of our LOCAL connector procedures like, "FillFromView()".
• SQL VIEWS
Where the data base engine handles the join and you treat multiple data tables as if they were ONE Clarion Table using one of our LOCAL connector procedures like, "FillFromFile()" or "FillFromSQLView()".
• CHT CLIENT/SERVER VIEWS
Where the server data engine handles the join and you again treat the data as if it were ONE Clarion Table - using one of our REMOTE connector procedures "FillFromUCRServer()" or "FillFromCHTServer()".
• SWITCHING BETWEEN CONNECTOR PROCEDURES
Since these "FillFrom" connector procedures are parallel, they can be generated individually from the template, two or more at a time, or all at once, and the design lends itself to readily, at run-time, switching between which connector procedure fills the browse, a LOCAL data connector, or a REMOTE data connector.
More in the next post... (More about Build 20D.02.00 Part 2)
We've just finished posting Build Update 20D.02.00 at 10PM, December 15, 2016 EST.
SIX new templates and lots of progress on ListBoxBrowseExtender. Look at HNDPEOPLE_LBX.APP (a new app) to see two LOCAL data procedures in action.
We'll do a longer writeup tomorrow on the forum and here on What's New?, followed up with a CHT Facts Sheet on the weekend or early next week.Enjoy!
This is a further progress report on the feature expansion of "LBX" (ListBoxBrowseExtender Template) as discussed here recently and in the near-past.
The overall objective with LBX is to easily build a Clarion browse in order to:
A) ... to be free of the ABC browse template, thus freeing the browse from a required direct attachment to the Clarion file drivers, while speeding up browse loading, querying and searching by an order of magnitude.
B) ... to use the resulting browse with local data (data tables located on the same machine running the Clarion browse app) or with remote data (data located on a remote machine potentially 100's to 1000's of miles away from the machine running the Clarion browse app). Switching from a local data source to a remote data source should not require a re-compile of the app, just a small re-configuration of the app (or just configuring the procedure) at run-time.
C) ... to work as a replacement browse in an existing app so that a traditional ABC app can have LBX procedures added into it which access data from a remote location while the bulk of the existing app continues, unchanged, to access data locally as before.
D) ... to provide built-in data transport encryption and compression. This is especially important when data are being transported long distances via the public internet.
E) ... to build a Clarion browse with "LBX" should be no more difficult, (ideally easier) than building it with the existing ABC browse template.
We've made significant progress in the last week or so moving LBX towards the above objective(s).
An LBX browse can access LOCAL tables, ISAM or SQL using ordinary Clarion file drivers directly.
An LBX browse can display browse data accumulated from no data file at all (data file free). For example displaying a disk directory or a list of files on disk (eg: HNDBULKSIGNER.APP) or it can display the contents of an ordinary text or code file (eg: HNDFBACKBUILDER.APP).
An LBX browse can access LOCAL tables, ISAM or SQL, using a CHT Client Server app as the data source, instead of Clarion File Drivers. (This uses Client Server concepts so it's two pieces, your app is the client and a CHT Type 3 Client Server is the data source).
An LBX browse can access REMOTE tables, ISAM or SQL, using a CHT Client Server as the data source. (This utilizes again, Client Server concepts, so it's two pieces, your app is the client and a CHT Type 3 Client Server is the data source).
Your apps can use mixed data sources, some browses filling from local data tables and some browses filling from remote data tables via a CHT Client Server.
An LBX-based application can share a remote (or local or WAN) CHT Data Server with other LBX-based applications.
We've so-far added three new templates (child, plug-in templates for LBX) that add various features into LBX. Various feature tabs on ListBoxBrowseExtender will explain how to insert one of these LBX plug-in templates into LBX to layer in an extra feature. (see next image)
Two of planned SIX new LBX tabs are now in place and are working in a no-embedding needed fashion. (see next image)
The FillFromView() tab, for instance, instructs you how to add a LBX feature-plug-in template called EmbedView_LBX.
An LBX browse can have all 6 fill methods enabled at the same time or a single fill-method can be enabled on the template if the LBX procedure is dedicated to LOCAL only, or REMOTE only.
Fill methods code is generated into the procedure only when a given fill method is switched on from the template. You can switch on one method, two methods simultaneously, or all six methods at the same time, and select between them at run-time.
FillFromFile() -- Used with a single local ISAM data file or multiple data files via an SQL view design
FillFromView() -- Used with multiple local ISAM or SQL data files
FillFromSQLSelect()-- Used with single or multiple local SQL data tables
FillFromCHTServer() -- Used with Remote CHT Client Server application local or remote, ISAM or SQL
FillFromUCRServer() -- Used with any CHT Server via UCR (User custom request). UCRs can be installed an all CHT Server types.
FillByEmbedding() -- Hand embedded fill method, for example reading a disk directory into the queue (requires you to code this, obviously).
More, as the work progresses....Gus Creces
While we work on revising ListBoxBrowseExtender (LBX), as already described in earlier posts on the topic, it will be useful to all interested developers to understand what capabilities CHT has already, by way of Client/Server templates, in order to understand how upcoming LBX features will mesh with both traditional Clarion browse building (on local data using file driver connections) and with CHT Client Server browse building (on remote data using CHT data Server connections).
As we said in an earlier post on the CHT forum, our current ListBoxBrowseExtender revision is complementary to CHT's already very extensive client server capabilities -- another way to interface browses and forms to the data server in a more informal way that readily lets the browse procedure act on local data (via SV file driver) or on local data (via CHT client server) or on remote data (via CHT client server).
The following document was introduced in 2014. It's a complete, extensively detailed explanation of how a remote data view is introduced to a CHT Client Server application and how to build a client-side browse for that view as part of a Clarion Client application built around CHT Browse templates containing no ABC Browse code.
We've given you two, already-working example apps, HNDMTSSV.APP (server) and
HNDMTSCL.APP (client) in your
/HNDAPPS/ directory. Any serious CHT developer
should be able to follow this document to learn the essentials of Client Server theory and
development using CHT Client Server templates.
If you have questions, feel free to ask them on the CHT forum as you work through this document step by step.
Should you have an interesting data design for a client/server app, that you'd like us to work up as an example, feel free to ask. You supply a dictionary and sample data, or part of a dictionary if there is a proprietary nature to the design that can't be fully shared with the CHT community. However, there has to be enough "meat" to the design to constitute a worthwhile example.
Or if you have a client/server project you'd like us to work through with you privately, at least until you find your Client Server bearings -- though this is not exactly rocket science -- ask CHT for a quote to pay for our time spent assisting you.
Here then, is the most recent revision of our Client Server document: Client Server Examples
To work fully, through the examples in this revised document, you need nothing new from us
in order to complete the learning exercise. Everything you need to start is already
there in your
/HNDAPPS/ directory including some example data tables,
a DCT and so on.
In the event you get stuck, ask on the forum and we'll give a detailed response that others may find useful also.Gus Creces
This flowchart illustrates how a ClientServerBrowseListBox desktop application is able to switch between data access points, whether local, or remote, from the same app, unchanged at run-time.
In an upcoming video, the same desktop app will be used to access sets of data tables located locally (on the same machine as the app) and remotely (on a machine 1000 miles away from the app).
How do we switch between data access locations?
We do this at run time by signing off one access location, changing the IP on the app's connection interface and signing on to the other location.
A video we'll be posting soon, illustrates this, as well as pointing you to a downloadable set of pre-built demo apps to let you try this yourself.
Stay tuned.Gus Creces
CHT Template Docs
CHT Application Docs (Complete)
CHT Utility Applications Docs
CHT Batch-Bot Application Docs
CHT Snap-In Application Docs
CHT Classes Docs
December 1, 2016