Sunday, April 22, 2018

Angular Asynchronous programming I - made simple

  One big advantage of using modern Javascript framework like Angular or library like ReactJS is asynchronous programming following Observable pattern. If you and your organization is OK with calling a heavy Soap Web Service and feel OK to see the mouse cursor spinning when waiting for the Web Service to respond back, no worry, you can skip this topic and continue with it. If you are fed up with such cursor spinning and want to see some snappy page rendering, read on...
  I will share with you my experiences on Asynchronous programming in the next couple of post. This post is a simple model using Observable pattern.
  First thing first, we need to define the event source - Subject. In Observable pattern, the source will produce the source event and data and broadcast it to subscribers who are interested in such topic and registered themselves as subscribers to such data and change event.

Step 1) Run ng command: ng g service message; this will create message.service.ts and message.service.spec.ts. Put following code in message.service.ts:

import {Injectable} from '@angular/core';
import {Observable} from 'rxjs/Observable';
import {Subject} from 'rxjs/Subject';

@Injectable()
export class MessageService {
  message: Subject<string> = null;

  constructor() {
    this.message = new Subject<string>();
  }

  /**
   * Send an message
   *
   * Ao Bie 12/09/2017
   */
  sendMessage(message: string) {
    if (message) {
      this.message.next(message);
      console.log('service sending' + message);
    }
  }

  /**
   * Clear message
   *
   * Ao Bie 12/09/2017
   */
  clearMessage() {
    this.message.next();
  }

  /**
   * Get message so that the subscriber can get a handle
   *
   * Ao Bie 12/09/2017
   */
  getMessage(): Observable<string> {
    return this.message.asObservable();
  }

}


What does sendMessage do, it is sending a string text message. Once sendMessage gets called, the subscribers will be notified of this new message.

Step 2) Put below logic in your component ts file:

import {Component, OnInit, OnDestroy} from '@angular/core';
import {Subscription} from 'rxjs/Subscription';
import {MessageService} from '../message-service.service';

@Component({
  selector: 'app-header',
  templateUrl: './header.component.html',
  styleUrls: ['./header.component.css']
})
export class HeaderComponent implements OnInit, OnDestroy {
  message: string = null;
  subscription: Subscription;

  constructor(private messageService: MessageService) {
    this.subscription = this.messageService.getMessage().subscribe(
      message => {
        console.log('header got message: ' + message);
        this.message = message;
      }

    );
  }

  ngOnInit() {
  }

  ngOnDestroy() {
    this.subscription.unsubscribe();
  }

}

In the constructor, we inject the newly created service instance and subscribe this component to the subject in message service. When you subscribe to the Observable, you also provide your call back logic to handle the new data when being notified of latest chage. Above block in bold is the call back logic.

Be sure to unsubscribe in ngOnDestroy, otherwise you will observe memory leak when you run your web application.

Step 3) in another component ts file, catch the value entered from UI and broadcast this new value:

import {Component, OnInit} from '@angular/core';
import {MessageService} from '../message-service.service';


@Component({
  selector: 'app-work-pane',
  templateUrl: './work-pane.component.html',
  styleUrls: ['./work-pane.component.css']
})
export class WorkPaneComponent implements OnInit {


  constructor(private messageService: MessageService) {}

  ngOnInit() {
  }

  obSubmit(form: any) {

  }

  /**
   * Enter Message
   *
   * Ao Bie 12/09/2017
   */
  onEnterValue(event: any) {
    console.log('onEnterValue: ' + event.target.value);
    this.messageService.sendMessage(event.target.value);
//    return false;
  }

}

Logic inside onEnterValue is using service object to broadcast the new change entered from UI. The latest data will be broadcasted and all the registered subscribers will be notified.

Just this simple, you will know what to do in the header.component.html if you finish this post and want to test it out.



Sunday, September 17, 2017

Angular4 experience


  After hearing from so many folks talking about Angular, finally got some serious project experience on it. Our organization is currently a heavy Oracle ADF workshop. Most of our complex business applications are written in ADF. So far only one of my ADF application is the fastest and most performant one in all these ADF applications. In terms of business complexity that performant application is not the most complicated one. But it also covers a lot of area: 
    1) filtering data through large data set (hundred or thousands records), present the filtered data for update;
    2) creating new records;
    3) clearing the filter;
    4) search form;
    5) mass update based on search result in 4), but first of all save the data before actually update;
    6) upload and download large data set meta data table;
    7) flexible access control;
  There are some effort outside of ADF needed:
    a) make sure each column that has a filter defined is properly indexed;
    b) some forward only view object created and Apache APIs called to avoid large memory footprint during downloading and uploading chunky data (>10,000 records);

Then with all the ADF best practice being applied, what we can achieve is fast individual page and page components. I still notice that initial application loading time is long (depends on what has been put in Application Module prepareSession and home page on load server listener), sometime for no reason the page navigation could be also too long - mouse input becomes circle and spin for seconds at least...
  After completing the main flow of rewriting one of our internal ADF application using Angular4, I felt that I was amazed by how fast the new application can be and shocked by the fact that I have spent so many years enjoying being an ADF expert and that a good framework like ADF which I loved and was proud so much becomes so easy to be put down by things like Angular...
  AugularJS is MVC of Javascript on practice, ADF is MVC of Java on practice. And what makes ADF page always seem to be too heavy, and yet as of now still not able to support HTML5 completely. I think many of these questions and doubts can only be answered by Oracle if Oracle is willing to share.
  One thing I know so far is Angular is really capable of doing some very serious enterprise calculations or logics without question.
  One advantage when comparing JS framework with ADF is JS is directly fit into browser and is the native language of browser. So it can easily manage any server state with the help of LocalStorage. Whereas in the traditional client server mode the state management involves the server and hence the overhead will be too high in terms of navigation control happens. I think not much to post to server to store the states in Angular.
  Also Angular is more MVC'ed since Angular2 which makes Angular stand out of ReactiveJS and Oracle JET. Data binding is also made easy since Augular2 and developer can really focus on business logic and make the entire application or individual modules well designed and loose coupled - all just simple and easy. Of course Rest Service - part of the business model also plays a very important role and actually a key role in Angular application development, if you want to be a full stack Angular developer.
  I will share with more in my following posts, stay tuned...

Saturday, June 11, 2016

ADF View Object Performance Tuning

  Recently there is a use case which is very interesting. We have an Enterprise web application which supports multiple countries. For one country it specifically has something called Contributor Class. It is comprised of more than 8000 static records, and needs to be displayed on UI page as an ADF LOV. The initial design and implementation of this LOV from another developer was to use a transient view object. Then the developer overrides transient view object's life cycle methods:
     executeQueryForCollection(),  createRowFromResultSet(),  hasNextForCollection(), getQueryHitCount(), findByViewCriteriaForViewRowSet()
  What I derived from such implementation is the original developer wants to achieve case insensitive search in the UI LOV. Because I saw such case insensitive logic in overridden findByViewCriteriaForViewRowSet method.
  Later on another developer implemented some logic in web service layer to default a value for contributor class on UI page when a member id is entered.
  Due to corporate security policy I cannot disclose source code here, and no screen shot either.
  To help you understand, I am re-iterating all the facts again below:
  1. Order capture enterprise web application has an ADF LOV - (Contributor Class) on UI page backed by an ADF transient view object;
  2. A web service was implemented to default the Contributor Class LOV at run-time when a member id is entered;
  3. Order capture web application has a header section which has a shipping information section and Contributor Class LOV is part of it; the shipping information section ADF iterator binding has 'ppr' as its change event policy;
  Then after these 2 new features went in production, user reported that when they a) enter a member id, a contributor class was defaulted; b) they found the defaulted contributor class is not correct, they corrected it; c) they priced the order; Bang! These 3 simple steps took 11 minutes to finish. Our production server is a powerful Linux multi-CPU with multiple virtual servers for our order capture application.
  Why?
  This contributor class is an ADF input LOV backed by a transient vo, and its parent view object is shipping address view object. In page definition file its change event policy is ppr. So any other shipping field change will cause it to refresh - to re-execute its query. By default since it is programmatic view object, there is no actual database SQL query, its view cache will be cleared when executeQuery() is called and the framework will repeatedly call createRowFromResultSet in its ViewImpl class to re-create 8000 rows again. On UI, when end user clicks on LOV amplifier glass, or enters a value it will take more than 30 seconds for the LOV search criteria pop-up to come on screen. User browses and selects a value from 8000 rows will also be painfully slow. User makes a selection and closes the pop-up will take 30 seconds again each time.
  Enough story, what I did to address it?
  1. Create a database table, build Entity Object; Check 'Use Update Batching' in Entity Object 'General' tab 'Tuning' section. This option doesn't help much in this case
  2. Create View Object from Entity Object; 
  3. Create 'insert only' version View Object from Entity Object; JDeveloper View Object editor 'Overview' 'General' tab 'Tuning' section 'Retrieve from the database' section, choose 'No  Rows' (i.e. used only for inserting new rows). 'Access Mode' choose 'Forward Only'. This step is used for programmatic populate contributor class view object from web service. Since it is forward only view, framework doesn't need to care about run-time scrolling back and forth within the retrieved data set. Since 8000 records uploading into database will have a lot of overhead, this actually will be slower than populate transient view object. But this forward only view object will be the fastest for populating a view object backed by an entity object.
  4. Create view criteria for view object created in 2; the criteria item will by default support case insensitive search. Also notice that query execution mode is 'In Memory'. 
  5. In parent view object, specify the search region to use view criteria you created in step 4. which supports case insensitive search;
  6. Define SQL query hint for view object created in 2;  this step is very important. We already know that SQL query hint will improve SQL query drastically. This tells SQL parser what you intend it to do. In this case the SQL parser will do its best to populate run-time UI  view port with records retrieved from database in the fastest way it could.
  7. Generate database loading SQL scripts to populate database table; This way we can skip step 3. If using step 3. we will put some initial loading logic in ApplicaitonModuleImpl prepareSesion() method to do so. And such initial loading of 8000 records will take about 45 seconds when using Entity Object backed View Object in Forward Only mode. Why do we need to bear with such time taken loading? We will be better off populating the database table and making sure such table will be synchronized every once or twice a year when government updates contributor class list.
  That's it, by doing the above the performance issue was addressed nicely. The waiting time in production was reduced from around 10 minutes to a couple of seconds.
  There might be other approaches, like change transient view object searching mode so that executeQuery() won't clear view object cache. I might try such approach in the future. But it never harms to use standard approach recommend by Oracle - create Entity Object and updatable View Object to take advantage of entity object cache. They are reusable business objects after all!

Sunday, June 14, 2015

An implementation on automatic data push in ADF using Active Data Service (ADS)

  From time to time when we develop applications, our application is not running on its own. Chances are the application is always interacting with other applications. 
  Let's assume you have this requirement for your application: your application has a page to display data from a database table, but another application will update the same table from outside of your application's context. What is the best approach to synchronize your application with the data that are modified from another application?
  There are the following ways to achieve such requirement:
    1) Your application keeps waiting on possible data change; this is not recommended, because your application acts like it is blocked, it won't respond to other requests. Use this approach if you have to. Like your application has to wait for certain change to proceed. For example your application is a payment gateway, it redirects your end user to a bank page for the end user to do some online transaction. Your application has to wait for the end user to finish and process the bank transaction response in your application.
    2) Your application add a 'Refresh' button onto your page, ask your end user to click on this refresh button to refresh data update onto your screen; using this approach the user experience is largely relying on if end user is OK to keep pressing button to refresh application data.
    3) Your application initialize a thread to poll the database table at certain frequency ( for example poll database table every 30 seconds ); this approach will save end user from keep clicking 'Refresh' button, but the down size is your application might have to refresh page each time the poll happens. Some user might complain that your page flashes or flickers. Another big drawback is polling database may cause database performance issue.
  4) Your application continues after displaying data; you will still have some logic after the data changes are pushed to your application and your application can process such data changes at the back ground while still servicing other requests.

  As you can see approach 4) above using automatic data push gives your application more flexibility in a cleaner way. Every other approach might be OK in a case by case scenario. 
  Fortunately there is a simple approach to achieve auto data push in ADF using Oracle database and ADF Active Data Service - ADS technology. The following are the steps to implement this from scratch.

Grant 'change notification' to your database user
  In my sample application I am using scott database user, log into your database and issue the following SQL command:
    grant change notification to scott
  You need to grant such permission as sys or system or any user having DBA privilege.

Code your database change listener
  Your ADF code must have a database change listener registered to listen on the backend database table. The following code snippet is an example:
    private void startListenForDBChanges() throws Exception {
        OracleConnection conn = connect();
        Properties prop = new Properties();
        prop.setProperty(OracleConnection.DCN_NOTIFY_ROWIDS, "true");
        dcr = conn.registerDatabaseChangeNotification(prop);

        try {
            dbChangeListener = (new DatabaseChangeListener() {
                    public void onDatabaseChangeNotification(DatabaseChangeEvent dce) {
                        String rowId =
                            dce.getTableChangeDescription()[0].getRowChangeDescription()[0].getRowid().stringValue();
                        System.out.println("Changed row id : " + rowId);
                        String changeDetail = formChangeDetail(rowId);
                        listener.onDBChangeListener(changeDetail);
                    }
                });

            dcr.addListener(dbChangeListener);

            System.err.println(dcr + " " + dbChangeListener);

            Statement stmt = conn.createStatement();
            ((OracleStatement)stmt).setDatabaseChangeRegistration(dcr);

            ResultSet rs =
                stmt.executeQuery("SELECT ename, job, sal, comm FROM bonus");

            rs.close();
            stmt.close();
            conn.close();
        } catch (SQLException ex) {
            if (conn != null) {
                conn.unregisterDatabaseChangeNotification(dcr);
                conn.close();
            }
            throw ex;
        }
    }

  In the above code, the "SELECT ename, job, sal, comm FROM bonus" is to let your code listen on ename, job, sal and comm columns of bonus table. Any changes related to thse 4 columns will be intercepted by your database change listener.

Construct your change notification message
  Your can have a method to construct the change notification message that might be displayed onto your application page.

    private String formChangeDetail(String rowId) {
        PreparedStatement stmt = null;
        ResultSet rs = null;
        String ename = "";
        String job = "";
        BigDecimal sal = null;
        BigDecimal comm = null;
        OracleConnection connection = connect();
        try {
            stmt = connection.prepareStatement("SELECT ename, job, sal, comm FROM bonus WHERE ROWID=?");
            stmt.setString(1, rowId);
            rs = stmt.executeQuery();
            while (rs.next()) {
                ename = rs.getString(1);
                job = rs.getString(2);
                sal = rs.getBigDecimal(3);
                comm = rs.getBigDecimal(4);
            }
        } catch (Exception ex) {
            ex.printStackTrace();
        } finally {
            try {
                rs.close();
                stmt.close();
                connection.close();
            } catch (Exception ex) {
                ex.printStackTrace();
                System.err.println("Cannot close connection.");
            }
        }
        return "Emp name: " + ename + " job: " + job + " salary: " + sal + " commission: " + comm + ".";
    } 

 Code your AdsController class
  Your AdsController java class should have this declaration:
    public class AdsController extends BaseActiveDataModel implements DBChangeListener {}
  You will register your AdsController class in your application same as you do as backing bean class.
  
  Add active model statement in your AdsController constructor method:
    public AdsController() {
        ActiveModelContext vActiveModelContext =
            ActiveModelContext.getActiveModelContext();
        Object[] vKeyPath = new String[0];
        vActiveModelContext.addActiveModelInfo(this, vKeyPath, "Status");
    }
  Important: "Status" is the name of your change notification event name. When I was first implemented ADS, I was really confused by what the 3rd parameter means in addActiveModelInfo(), it is just a name. You should use such name consistently in your application. Later in triggerActiveDataUpdateEvent() we will use this same name again. You need to make sure they are the same.

Override startActiveData method in AdsController
    protected void startActiveData(Collection<Object> coll, int i) {
        System.err.println("Starting active data thread for: " + this);

        dbPushController = new DBPushController();
        try {
            dbPushController.addChangeListener(this);
        } catch (Exception e) {
            e.printStackTrace();
        }
    }

  In the startActiveData method, you add change listener into your ADS controller class.
  
Override onDBChangeListener method in AdsController
    public void onDBChangeListener(String changeDetail) {
        triggerActiveDataUpdateEvent(changeDetail);
    }
...
    public void triggerActiveDataUpdateEvent(String changeDetail) {
        counter.incrementAndGet();

        System.err.println("Trigger active data update event: " +
                           counter.get() + " " + this);
        ActiveDataUpdateEvent event =
            ActiveDataEventUtil.buildActiveDataUpdateEvent(ActiveDataEntry.ChangeType.UPDATE,
                                                           counter.get(),
                                                           new String[0], null,
                                                           new String[] { "Status" },
                                                           new Object[] { changeDetail });
        fireActiveDataUpdate(event);
        j++;
    } 
 

  Since we declared AdsController as implement DBChangeListener, we have to implement onDBChangeListener() method, in it we fire the active data update event. Notice "Status" in bold in triggerActiveDataUpdateEvent() method, it is the same data change name we used in AdsController constructor method.

  That's pretty much it. You need to define a connection in your ADF application model project. I am using scott and bonus table for this sample.
  Please see the screenshot of running this application below.

When application first runs, no data in scott.bonus table yet


Now add a record for SMITH and give SMITH 1.5% commission

Now modify SMITH's commission from 1.5% to 2%



At last we add 5% commission to Jones



  Congratulations! You now know all it about to implement Oracle ADF automatic data push via ADS and database change listener! 
  Please use the following link to download the source code of this sample:
Data Push Sample Source Code 

   

Friday, June 5, 2015

How to implement 100% Dynamic ADF table with inputText having value change listeners.

  I have a use case that has the following business requirements: 

    1) Have multiple database tables and display them on the page; 
   2) The application needs to have the capability to track end user changes;
  
  The easiest way in ADF is to create the number of Entity Objects and same number of View Objects. Then user needs to create an ADF page and drag drops the same number of times on the page. After that user needs to have a drop down to allow end user to choose which table to work on and based on this selection the corresponding table will be rendered and all the others will be hidden. User needs to define a value change listener for every field for each table.
  This solution seems OK for a handful of tables in the application, how about the developer needs to work with a subset of Oracle E-Business Suite Inventory module tables which can be several tens of thousands of tables!
  There is a better solution. Use dynamic view, dynamic tables. Along with my implementation, I referred to the following blog post: Eugene Fedorenko's ADF Practice blog - "Dynamic Table Declarative Component" and Andrejus Baranovskis Blog - "ADF Generator for Dynamic ADF BC and ADF UI". Engene and Andrejus has given a working sample respectively. My job is to understand the code and put them together, extend and tweak to meet my specific requirements.
  The following are the break-down of the design and implementation steps towards the completion of finishing this application.

 Choose refreshing mechanism
  No matter what approach you select, you will have a drop down containing all the table names. Then after each selection the table display area needs to be refreshed. One can put everything on a single page and use ppr appropriately to refresh. Or one can put table display area in a region. Such region has a Bounded Task Flow behind it and the BTF has its refresh set to 'ifNeeded' in page definition file and when its parameters change, the BTF will be automatically refreshed. In my sample I am using the latter.

 Dynamically construct view object
  The constructVOInstance() method in AppModuleImpl.java file is the one which construct the view object at run time on the fly. It retrieves all the Entity Object's attribute names and define the view definition file on top of them, then calls createViewObject() to create the view object.

         ViewDefImpl viewDef = new ViewDefImpl(viewDefName);
            viewDef.addEntityUsage(shortEoName, eoName, false, false);
            viewDef.addAllEntityAttributes(shortEoName);            

            String select = ViewDefImpl.buildDefaultSelect(viewDef.getAttributeDefs());
            viewDef.setSelectClause(select);

            viewDef.resolveDefObject();
            viewDef.registerDefObject();

            String from = viewDef.buildDefaultFrom();
            viewDef.setFromClause(from);
            System.out.println("vo Name : " + voName);

            ViewObject vo = createViewObject(voName, viewDef);


 Build your dynamic table on page
   Developer needs to drag the DummyVO onto his/her page/fragment and drops it as:
    a) ADF dynamic table with dynamic:table tag; 
    b) ADF dynamic table with af:table tag; 

I later found out that dynamic:table is not applicable in my case because developer cannot add anything inside it. So here is what I am using on my page fragment as the definition of ADF table:
    <af:table value="#{pageFlowScope.TableBean.tree.collectionModel}" var="row" rows="#{pageFlowScope.TableBean.tree.rangeSize}"
                      emptyText="#{pageFlowScope.TableBean.tree.viewable ? 'No data to display.' : 'Access Denied.'}"  fetchSize="#{pageFlowScope.TableBean.tree.rangeSize}" rowBandingInterval="0" id="t2">
                

                <af:forEach items="#{pageFlowScope.currentViewDefs}" var="def">
                <af:column sortProperty="#{def.name}" sortable="false" headerText="#{def.name}" id="c1">
                        <af:inputText value="#{row[def.name]}" id="it2" autoSubmit="true"
                                      valueChangeListener="#{mainBean.fieldValueChange}"/>
                    </af:column>
                </af:forEach>
            </af:table>

  We will deal with the tricky part later, but basically I am achieving dynamic by embed af:forEach underneath af:table to get dynamic.

  Notice that the "value" attribute has something like: #{pageFlowScope.TableBean.tree.collectionModel}. This is one of the tricky part. The dynamic is coming from the logic in backing bean getTree() method.

 Construct JUCtrlHierBinding dynamically
   You see what is in the getTree() method in TableBackingBean class.
       JUCtrlHierBinding chb = (JUCtrlHierBinding)dcb.findCtrlBinding(TREE_NAME);

        if (chb != null) {
            dcb.removeControlBinding(TREE_NAME);
        }

        //Looking for the VEmpIterator iterator
        JUIteratorBinding iter = (JUIteratorBinding)getIterator(ITERATOR_NAME, VO_NAME);

        //Create and init a tree binding definition
        JUCtrlHierDef hierDef = new FacesCtrlHierDef();
        HashMap initValues = new HashMap();
        initValues.put(JUCtrlHierDef.PNAME_IterBinding, iter.getName());
        JUCtrlHierTypeBinding typeBinding = new JUCtrlHierTypeBinding();
        initValues.put(JUCtrlHierDef.PNAME_TypeBindings, new JUCtrlHierTypeBinding[] { typeBinding });
        hierDef.init(initValues);

        //Create a tree binding instance
        chb = (JUCtrlHierBinding)hierDef.createControlBinding(dcb);

        //Add the instance to the current binding container
        dcb.addControlBinding(TREE_NAME, chb);   

   Same as when constructing ViewObject dynamically, when you try to construct the JUCtrlHierBinding, you need to construct its Defition first.
  You may already know that the corresponding Java class for af:table is RichTable and inside it there is something called wrapped data, which is the collectionmodel normally in af:table's value attribute. And it is the data set for af:table's var attribute to iterator through such data set. This wrapped data or collectionmodel has its corresponding Java class too which is JUCtrlHierBinding. These are the page UI layer objects, and at the page definition layer there is Iterator Binding or JUIteratorBinding Java class. And down the chain at the ADF Model layer it is ViewObject in Application Module or ViewObjectImpl Java class. These are pretty much what Oracle ADF data binding is all about and how the data flows during data binding. 
  Lastly I drag drop the TableNamesVO onto my main page as a Select One Choice. This view object contains the SQL query for all the table names that need to be managed in this application. When the value of this drop down changes, the table name will be the parameter for the table display area bounded task flow and cause it to be processed as illustrated in 'Dynamically construct view object' section.
  In this sample I am using Oracle's scott schema. There are 4 tables from scott schema will be shown in the table selection drop down. In order to correctly run the sample application, please make sure scott schema is unlocked and is connectable. See the following screen shot for the resulting page:
 
  If you need to add another 100 tables to be managed for this application, all you need to do is make sure TableNamesVO SQL has the new 100 table names, that's it!
  These are all you need to know in order to have a very dynamic, flexible, extensible and sustainable approach to achieve 100% Oracle dynamic table.
  Please find the working sample code below in the attachment of this post. Enjoy! 

 DynamicEditableAFTable.rar