Monday, March 16, 2015

Data driven programming techniques: strength and weakness.

Data driven programming techniques: strength and weakness.

Some time ago I was able to make a whole inventory and shop application from a scratch in drastic 3 month timeframe. I guess the next time it could take 1/3rd of it. One of reasons for such speed was in the use of data driven UI and web services. The database schema was a source for web services configuration which was loaded along with the web application. Than it was used for data type operations (casting, validation, etc.) on web service level and for UI binding on front-end.

The magic is simple: each data type was presented as a read-only and writable UI widget. In such way where shown the primitive types, complex types been presented as tables with a common parent to serve a generic fields containers. It was a rare case of overriding of specific to the type functionality. Same concept of 90% web services implementation used common base functionality with adjustments only for 10% of special cases.

In parallel for same company has been made another data-driven forms framework which had a hiccups all the time. The ratio of special cases vs. reused code there was in reverse: 10% of common, 90% customization.

While both “frameworks” shown significant advantage in comparison to usual case-by-case development eventually the higher customized one started to fall apart: the development became less manageable and almost touched the same level of complexity as usual single form development practice. Looking back the many its aspects could be designed deeper which will make a better impact. But the major problem still will be unresolved: under data driven patterns in most cases the Software Development Life Cycle (SDLC) will be broken. Data driven UI usually introduced as a shortcut for long lasting case-by-case development. And as such missing many cycles, before enforced either by process or by development environment: from design (why we need a design if it is a part of metadata?) to versioning, code review up to release approval. Such systems in agile environment often developed and run in production resulting in emergencies and fast patches rather thought-through solution. Once the primary development is done it is assumed that maintenance and code adjustment will be easy, the work on changes is relocated to less experienced developers. 

Without restrains of usual SDLC it could be a disaster. So to keep it under control all parts of SDLC need to be a part of this data-driven framework. Without it the scope insulation of later changes and simple roll back process are the bare minimum.

In the inventory and shopping cart application above the DB schema was serving as metadata source for:
·         Database tables exposed as web services
·         Forms based UI
Does metadata have more use? Of course. J2EE developers immediately will recall the service flows, Object to data model mapping and many other cases, usually defined in XML files on project. Those are so deeply integrated with Java environment that people taken it as given.

I would like to bring you back to UI which modern days also taking over lot of business logic to its side. The fat client relives the server not just from the UI rendering but also serves as a gateway to the web cloud services. That is IMO Web 3.0 concept: the client is an application on its own and became a nerve cell in collective distributed network brain with server connections as synapses. The server is not in charge of whole process anymore, clients live and interact on its own discretion.

That idea defines the complexity we will face in fat client apps. The industry is trying to manage the scope by introducing so called “one page” applications. In another word by splitting the complex tack on primitive ones which do not have any or just little smarts on it. I guess that explains the popularity of AngularJS and its competitors.

But sometimes it is a good advantage to have a fat client app or module to leverage the cross-site knowledge and provide appropriate user experience. In my case the sync between version control, Wikipedia hierarchy, documentation sites and test results take too much orchestration if implemented via web 1.0 or web 2.0 patterns. The sequence of hierarchy discovery, sync data with automated and interactive merges, progress updates and error/retries handling do not fit easy into web service – UI pair. Old dogs would advise to look into server-side model with live sync like IceFaces or Vaadin. Modern ones will refer to MeteorJS. While that could be doable (not sure) the idea of keeping live model on server side is hardly scalable. Few hundreds simultaneous sessions will need proportional resources increase on server side. Rather the fat client will use just a bare minimum from each involved server.

Programming of complex apps is not easy a priory, on front-end especially. The XHR chain for the described tack with complex hierarchical orchestration is impossible with usual callbacks: too many of them will be encapsulated into each other. The deferred pattern given a bit more flexibility on scenario creation. At least sequential chaining is a part of when().then()

Such pattern could use a help from frameworks. Dojo Toolkit given the deferred collection (dojo/promise/all); some essential deferred combinations you could build yourself (like the dynamically added dependencies during hierarchy traverse via XHR). It still will be the hardest challenge to read and maintain such code. Not to mention to give such to third party developers for own modules integration.

The data driven flow will be the answer for resolving of complexity above. It matches the aspect oriented or reactive programming principles: the model change is handled by small and discrete actions. You could name it data change event handlers if that is easier to understand. In such case the flow will be defined by current data state. This state could be preserved and recovered at any moment giving ability to develop and test the small steps independently simplifying the development.

The decision on what form the metadata presented is on you. Handy part for it would be the hierarchy query, ability to persist/(de-)serialize, mix and separate different data aspects on same level of hierarchy( aka namespaces), change event subscriber API. Guess what has been chosen by XML guy J

Happy coding!

Wednesday, March 11, 2015

Web 3.0?

Modern days web page is also taking over lot of business logic to its side. The fat client relives the server not just from the UI rendering but also serves as a gateway to the web cloud services. That is IMO Web 3.0 concept: the client is an application on its own and became a nerve cell in collective distributed network brain with server connections as synapses. The server is not in charge of whole process anymore, clients live and interact on its own discretion. That idea defines the complexity we will face in fat client apps. The industry is trying to manage the scope by introducing so called “one page” applications. In another word by splitting the complex tack on primitive ones which do not have any or just little smarts on it. I guess that explains the popularity of AngularJS and its competitors. But sometimes it is a good advantage to have a fat client app or module to leverage the cross-site knowledge and provide appropriate user experience. In my case the sync between version control, Wikipedia hierarchy, documentation sites and test results take too much orchestration if implemented via web 1.0 or web 2.0 patterns. The sequence of hierarchy discovery, sync data with automated and interactive merges, progress updates and error/retries handling do not fit easy into web service – UI pair. Old dogs would advise to look into server-side model with live sync like IceFaces or Vaadin could potentially do. Modern will refer to MeteorJS. While that could be doable (not sure) the idea of keeping live model on server side is hardly scalable. Few hundreds simultaneous sessions will need proportional resources increase. Rather the fat client will use just a bare minimum from each involved server. Happy coding!

Sunday, November 2, 2014

AMD global require and relative path from HTML

In my environment there are lot of tests and templates are HTMLs with self-testing functionality. To make them path-agnostic I was using relative MIDs to same folder resources: scripts, templates and CSS.

ModuleFolder

  • MyModule.html
    • require(["./MyModule"],function(MyModule){ new MyModule({},createDivInBody() ).startup(); }
  • test/test.html
    • require(["../MyModule"],function(MyModule){ new MyModule({},createDivInBody() ).startup(); }
  • MyModule.js
    • require(["dojo/text!./MyModule.html","AMD/cssI!./MyModule.css"],function( template, css) {... }
  • MyModule.css
As dojo AMD and perhaps some other AMD implementations disrespect relative path on global require I need  either to work around or make AMD working as expected. By expected I mean not imaginary potential use of absolute path but real life expectation when looking on the code sniplet above.

There are 2 bugs exposed in the code:

  1. The repative path in HTMLs is treated not as module with JS extension assumed but rather abstract resource, so no JS extension added when path converted to URL. What it could be if not a JS and why there is a difference to MID ?
  2. In CSS plugin the relative path is not working either. It is computated relatively to dojo basePath
The bug fix is not an option(see bellow). What could be done?

  1. Shim the require() and rebase relative pathes with regard to existing basePath
  2. Create artificial "currentPage" package and place into require config, replace relative path dots with this package: require({paths:{currentPackage:location.href... }},["currentPackage/MyModule"]
  3. Use absolute package name: require(["lib/MyLib/PackageX/MyModule"
  4. give up on dojo and use another AMD loader
Perhaps someone could give a better suggestion which is quite appreciated.
If there is a solution, please share.

Regards,
suns

PS. Thank  Christophe Jolif's wisdom for this pain: https://bugs.dojotoolkit.org/ticket/14649
PPS. for interested will keep track on my blog.

JSON added to XmlView, a bookmarklet for web service presentation as sortable table

In addition to early promoted XML in XmlView the JSON format support appeared to be an easy effort. The recognition of JSON format and conversion to XML was all needed to make a sortable table presentation for JSON based web services.

It is not as efficient and scalable as XML though. JSON doubles the browser memory consumption, half of which reside in the JS VM; additional parser for JSON consumes extra CPU/time.

Is it possible to make a faster and smaller memory consumption implementation using JSON without XML?

I guess not. XML reside in browser memory and is not a subject for JS limitations which will be in place for JSON. There is no multi-threaded(yet?) template transformation from JSON to HTML. Not to mention there is no way to render directly DOM by template engine besides XSLT.

Potential for the improvement

There is a prototype for virtual scrolling via XSLT. It could be used as a base for rendering just visible part of data delegation the rendering on scroll in real time.

Happy coding!

Sunday, October 26, 2014

What are you after?

It is hard to guess what will happen with soul when the body is gone. There is no straight answer in any religion, at least I have not heard of it. But few guesses could be done based on some cross-religious claims.
All claiming that there is something for the soul, but what is a mystery.
Lets make a mind experiment and imagine what it could be.

It is definitely a stress to leave the body and huge uncertainty on what to do. Soul most likely would act according to its instinct which reflect its "true believes". Whether looking for angels, seeking own prophet or the guide to destination place. And believes will lead to matching destination. Self-estimation or guilt are those instincts which fulfill the expectations and eventually define the fate.
There are few scenarios most popular among believers. Soul of Buddhist will seek for available fetus to become own reincarnation. Others will seek for holy spirit which will guide to paradise of own kind or fear the devil on the way to hell. Atheists will be lost in the choice and could become poltergeist or ghost as not able to choose own way. Perhaps there is a chance that some demon or gang of those will try to catch the lost.

As "smart" homo sapiens I would like to be prepared and select the path which will allow my soul to grow, gain experience and carry some mission in long run. The minimal step is to develop a curiosity with scientific approach on same level of strength as true faith. It is essential to keep the options open in order to preserve the progress. Sticking to same scenario is circling in rounds. The external factors will affect the path and rather following own agenda the stronger entities will be able to manipulate and use the soul. They also will try to keep them circling until reached wanted state or worn out.

Curiosity itself is not a path or agenda. The nearest and a step or two ahead plan would be handy. High-level agenda is declared earlier, how to achieve it?
Only known world is this one and potentially something in the middle. In this world we growing, getting enlightened or reverse. At least the progress here could be achieved for sure. So it would be logical to try to get back. One of declared ways is reincarnation. There are few others but less popular.
The self-improvement could be done on "carma" level. Which is unconscious and not reliable, most likely breakable by other stronger souls. Or on sane level. Which requires previous incarnations memory recall or at least same on soul stage. It has been claimed that most enlightened people could reach the level when previous lives recalled. Hmm, does not seem an easy option...
Are there other ways? Perhaps. There is a popular concept of conversation with dead people. Perhaps it would be the way to speak to yourself also? Seems it will require as gaining of whole xxx skills and ability of identifying yourself on top of it. Such approach would be quite dangerous to expose in modern society as not welcomed by any religion. On opposite the enlightening could be advertised and the help is much expected.

There is an effort associated in achievement of memory recall. As been read at least a decade to reach some approximation to "nirvana". Does it include the incarnations reunion in same consciousness or some more work need to be done? Adding the question to my todo list.

Despite of answers on questions above, the far goals of eternity should be leading to same conclusions as real life goals realized only inside of this world among live people. Be curious, learn and adopt knowledge, seek the path, state the goals and reach them.

XmlView browser button shows XML as a sortable table

XmlView

The one extra module appeared on XmlAspect.org. This is a browser button to make the web services visualization. Along with Web 2.0 methods implemented by client-side cross-browser XSLT rendering it is a great showcase for CSS3 capabilities. The collapsing and sub-branch selection is implemented by pure CSS without any JS.

While JSON fans have been claiming the XML presentation is ugly and not useful, I was happy with its default presentation as a collapsible tree in browser. But most of web services comprise the tabular kind of data. Which is more helpful when shown as a table. The sorting on all columns could help with usability. That is what the tool does: present tabular data as sortable table.

Besides changing the sorting the URL will be adjusted by adding the sorting parameters into hash part. That will allow to share the URL with view parameters set.

As a tool for developer the cross-browser support will rely on audience interest. For now it is tested on current Chrome and FF.

The module is completely open sourced, feel free to use anywhere. Github given a great service where I am happy to use SVN on git repository.

Happy coding!

Saturday, June 7, 2014

CSS3 trigger UI

How to make trigger UI using only CSS?

Once in a while there is a need to have some kind of trigger for checkbox functionality on the plage. Usually it is a regular <input type="checkbox"/> styled with background images like those: .
To use the styling based on background image you would need JS as INPUT itself does not have the ability to override the image on its container level during change event. There are some tricks how to do that properly as you do not want to use JS during page load to make the checkbox UI.
CSS3 gives you ability to implement the trigger UI design without any JS. The magic reside in ::before selector and background: linear-gradient. It is backward-compatible and shows regular checkbox where CSS3 is not available.

input[type='checkbox']{ display: block; width: 100%;margin:0; }
input[type='checkbox']::before
{ content: "OFF";
  display:block; padding: 0 1em 0 60%; color: white; font-weight: bold;
  text-align: "right";
  border: 1px solid black; border-radius: 3px;
  background: #73777b; /* Old browsers */
  background: linear-gradient(to right, #ffffff 0%,#ffffff 49%,#f1f1f1 50%,#73777b 51%,#73777b 100%);
  white-space: nowrap;
}
input[type='checkbox']:checked::before
{ content: "ON";
  padding: 0 60% 0 1em;
  background: #49aa50; /* Old browsers */
  background: linear-gradient(to right,  #49aa50 0%,#49aa50 49%,#f1f1f1 50%,#ffffff 51%,#ffffff 100%); /* W3C */
  white-space: nowrap;
}


Links: gradient-editor
Happy coding!