CHT (Clarion Handy Tools)
Templates, Classes, Utilities, Apps, Projects

June 1, 2020

CHT 2020 VERSION 24B.01.00
BUILD HIGHLIGHTS

We've now posted the next CHT build update numbered 24B.01.00.

REVISED DEMO APPLICATIONS

Reinstated Demo Application: HNDSMSMESSAGE.APP

This "CHT Utility Application" illustrates both CHT Native SMS messaging as well as SMS messages via SMTP. This application requires Clarion's CLARUNEXT.DLL but is otherwise stand-alone.

Further it is dependent on correct configuration of a valid SMTP server or SMS gateway setup with UserName, Password, Email/SMS Server and Email/SMS Return Address. Our template and SMS classes provide a configuration function with user-controllable registry location setttings.

The SMS template illustrated is EMBEDSMSFUNCTIONS. The CHT class underlying this template is HNDSMS located in files HNDSMS.INC and HNDSMS.CLW.

Please read and understand the proper formation of SMS addresses before experimenting, using this URL: SMS_GateWay

Reinstated Demo Application: HNDFBLOB.APP

This application illustrates two blob functions, JumpStartFileToBlob() and JumpStartBlobToFile(), available from the standard AACHTControlPanel template interface.

Please note, that the code in the EntryPointMain() procedure has very little to do with file to blob or blob to file. This code looks up a file name to be sent to a blob and manages the data base that stores the blob record. Your blob data base may look nothing like the file ZStore that we are using here.

The salient point is that when calling the functions JumpStartFileToBlob() and its inverse, JumpStartBlobToFile(), you are responsible for creating, fetching and saving the data base records in the blob-containing data table. The functions provided, will insert the files to and extract files independently from your blob record, data base I/O is up to you. However the I/O required is standard ABC stuff which you should already know how to do.

We have demonstrated here a couple of HNDDisk class functions to help us with our popup menu, when someone selects a file to be restored or deleted, for example. These have no direct relationship with the two blob functions being demonstrated.

REVISED JUMPSTARTS

These jumpstarts are located on the AACHTJumpStartProceduresUtility/2/3 interfaces on the "BULK", "ENCRYPT", "ZIP" tabs which display the following sub-tabs:

TAB: Bulk File Copying
** JumpStartBulkFileCopyWithProgress ()

TAB: Bulk File Moving
** JumpStartBulkFileMoveWithProgress()

TAB: Bulk File Deleting
** JumpStartBulkFileDeleteWithProgress()

TAB: Bulk File Renaming
** JumpStartBulkFileRenameWithProgress()

TAB: Bulk File Compression-Expansion
** JumpStartBulkFileCompressExpandWithProgress() -

TAB: Bulk File Containerization
** JumpStartBulkFileContainerizeWithProgress()

TAB:Bulk File Decontainerization
** JumpStartBulkFileDeContainerizeWithProgress()

TAB:Bulk File Encrypt Decrypt
** JumpStartBulkFileEncryptDecryptWithProgress()

WEB REQUEST FILTERING MECHANISM
CONTINUED FROM LAST MONTH

We wrote extensively about this topic in our "April" and "May" pages, so please read or re-read those articles, if you feel in need of a refresher.

From the release of our 24B.00.00 Version on April 14, 2020 to the release of this 24B.01.00 Version on June 1, 2020 we've greatly expanded the capability of our Web Filtering Features.

MUCH-EXPANDED CAPABILITY

Just as with polite conversation, client-to-server requests for data, pages, or information are based in mutual trust and respect for boundaries.

Unfortunately, these days, our commonly shared web neighbourhood is filled with overly nosey bullies. The behaviour of some, borders on outright thievery.

In light of this degenerating behaviour, the rules of polite and proper web discourse are out the window, and it is up to the server owner/manager to establish and enforce a set of rules which outline clear limits, using a server file called ROBOTS.TXT.

This file is designed to inform search BOTS where they can and cannot venture by listing subdirectories, file names and extensions and so on, which are off limits to them.

A digital NO TRESPASSING sign, if you like.

THE GOOD, THE BAD AND THE UGLY

Of course, for every "polite" BOT that behaves itself, there are dozens who brazenly ignore ROBOTS.TXT or use it as a "hit list" of places they might go treasure hunting.

This being the case, the ROBOTS.TXT file should make it abundantly clear that the rules, if broken, will result in consequences as follows:

#1 An immediate block is put on any rule-breaking request.

For example, there may be a rule to never enter a directory called "\admin\", whether such a directory exists or not.

#2 A temporary IP-block (SOFTBLOCK) is put on any IP that breaks any rule even once.

Duration of this temporary block is up to the server manager. The temporary block list is cleared when the server is reset. (See HNDSLFSV.APP)

#3 IP's breaking server rules repeatedly are added to a BLACKLIST

Henceforth, they are blocked (HARDBLOCK) from ever obtaining a server connection. Server managers are able to add or remove IPs from this BlackList at their own discretion. We've compiled an editable, BLACKLIST of approximately 1000 IP addresses with locations, based on our own experiences.

This is located in a file called: HNDDOSIPLIST.CLW (DOS = Denial Of Service). At present, our code loads a Clarion QEUEUE inside HNDDOSIPLIST.CLW.

In future CHT versions, the queue will be loaded from an interchangeable, filterable, past-behaviour-based data table.

We did it this way initially, so as to intimately familiarize ourselves with various intruders and their numerous errant behaviours. Blocked IP's in the blacklist have been painstakingly checked and rated against abuse registries.

However, the presence of any IP in our blacklist was ultimately triggered by repeated rule-breaking.

USER AGENT FILTERS

The list of server rules can include prohibitions on certain types of web BOTS, or USER AGENTS, known to be malicious or generally known to be of MAL intent.

You can see a list of these in a file called HNDUSERAGENTFILTERS.CLW These "User Agents" are simply refused a connection and are returned an error code.

Interestingly, once certain BOTs realize we are blocking them based on their use of a given "User Agent" name, they start sending HTTP headers with blank USER AGENT name fields or sometimes with all header fields blank. This is obviously the behaviour of entities with something to hide, so these IP's posting "BLANK" header requests are denied unconditionally, and IPs that attempt this ploy more than a few times are black listed for good.

WEB REQUEST FILTERS

The list of server rules can include prohibitions on certain server sub-directories file extensions, command lines, or execution attempts that are forbidden, and generally are totally useless against CHT Servers.

You can see a list of these, compiled from our server logs, in a file called HNDWEBFILTERS.CLW.

AT TIME OF WRITING

At time of writing we've gotten pretty good at detecting and deterring rule-breakers such that our fullfilled HNDREQUEST.LOG, contains 100% legitimate requests that have been successfully fulfilled and delivered.

To test this yourself, try our generic page server, HNDSLFSV.APP.It's ready to compile, run and test. With the "Log" switch checked it produces fourlog files:

HNDREQUEST.LOG (legitimate fullfilled requests)

HNDEXPLOITS.LOG (illegitimate unfullfilled requests)

HND404PAGE.LOG (legitimate unfullfilled requests, eg: missing or discontinued pages)

HNDUSERAGENTS.LOG ("USER AGENT" names used by various clients making requests)

Note that clicking the "Reset" button on HNDSLFSV.APP empties and backs up these log files and clears all "softblocks" while the BLACKLIST and WHITELIST IPs established in HNDDOSIPLIST.CLW remains intact.

The WHITELIST portion of IP addresses at the top of HNDDOSIPLIST.CLW represents known non-invasive bots such as GoogleBot (which indexes websites for the Google search engine).

Developers should also add their own IPs to this WHITELIST so that you can test, your server's blocking capabilities by breaking your own rules. Whitelist IPs are not added to the "SOFTBLOCK" list so that a legitimate request following a rule-breaking request is not automatically blocked, though it is still blocked for any single request that breaks server rules.

Contact Us

Click the link below to contact us by email.
It will start your email client with our email address inserted:

Click To Contact Us