mercredi 1 avril 2015

Version Control System

Let me ask you a question : "How do you manage your SQR files ?"

By how to manage SQR files I mean how to track modifications and modifiers to the SQRs.

Let me start with an example :

At one of my employers, there was only one Unix account every developer would use to access to the SQRs.  The rule was simply to login with this account, and modify the SQR; of course, people prefer to download it to their desktop, and then to upload it back to the server.  But one day, for some reason, someone forgot to download the current SQR to his desktop, took the one already present and overwrote the existing version on the server... that manipulation erased some changes that occurred since the first upload... Including tax updates improvement.  The company lost money because of this...

I propose to use SVN, a control version system, which is by the way used by other developers (Java / Web  developers) in the same department of IT.  But those developers, younger, work with new technologies, they document in a wiki, and use Scrum / Agile methodologies, whereas the Peoplesoft team is from old school.  At first, the boss is interested, but when she understands if a person really wants to commit without comparing with previous version and no-one makes checks after, the same problem may occur, she decided not to go forward.  She thought it would have been to difficult for "dinosaurs" to understand how to manipulate such a tool (even using Tortoise), and she didn't see the advantages to track modifications.

At another employer (or client), the method is relatively the same, except that they wanted us to rename a vanilla SQR with a prefix in front of it so their infrastructure team would see the SQR is custom.

At CGI, the SQR were managed by Stat, a migration tool... I found it complicated, but knowing the size of CGI, that may be the reason.

At my current employer, we have some paths that enables us to prioritize a custom SQR instead of a Vanilla one.  The SQRs are managed by Phire, a ticket software based on Peoplesoft, which is also used to migrate between environments.  It copies the SQR from one environment to another, but it seems complicated to track changes... and one day I noticed the infrastructure team put in prod a SQR not up to date... another story :)

Before working in Peoplesoft, I worked in PLM (Product Lifecycle Managment), which was C++ development to be short.  I worked at Boeing, and code was really well managed.  Boeing purchased a tool named ClearCase, which would track all of the sources and properties files.  Before modifying a file, we had to reserve it; of course, we could work on files without reserving them, but when the time to commit changes came, we could have to manage potential merges with the developer who had reserved the file.  It was always easy to know who modified what.  We were part of a large team, at least 30 developers, so there was a CleareCase administrator who would deploy different branches of the repository in different developing/testing environments.  Everything worked like a charm, even if it was a bit complicated at the beginning.

To be continued...

Another Horror Story

I'm still working for the same client that I won't name.  I was asked to debug a program delivered few years ago by "Expert Consultants" from huge firms I won't name either.

So I start looking at the appengine... find the Peoplecode where the file is generated... OMG : 1953 Lines.

Well I start with a quick look, I scroll up and down quickly... I notice something strange : some blocks of lines repeated, but idented to another level at each time.


I look closer, try to understand... I finally understands : a record ( &RECORD1 ) contains some fields named the same way (a number is distinguish them) and having the same usage.   There are 23 clones same 5 fields in the record.  So 23 if statements.

I finally understand the context : the code is part of a While statement.  At each iteration, the program checks the iteration number (stocked in a variable named &n_loop_2) and set the values of the fields ending with this number.

Do expert peoplesoft programmers, in particular those who charge 200$ an hour, ignore that we can access to a record or a field (or other objects) by stocking his name in a variable ?  Can't they search a little bit in Peoplebooks ? Can't they try to find a way to do stuff dynamically instead of copy-paste ?  I discover that when I had less than a year of Peoplesoft background... maybe I'm too curious... :)

Ok, I continue looking... I found 2 more times the EXACTLY same issue; The little difference is that a letter differs in the fieldnames of the 3 blocks :

ZZ_zzz_G_030_A_xx
ZZ_zzz_G_030_B_xx 
ZZ_zzz_G_030_C_xx

With this new discovery, I decided to investigate more... I found out that there is another while at an higher level.  The student (we are in Campus Solutions) can have 3 differents plans, for each plan, there is a block of code... I separated the three blocks, and compared them (thanks to Beyong Compare). The results are crazy : the block of code for B is identical to the block for C and A's differs just a little bit. Only suffix differs.

So I start refactoring... There is no way I will try to understand such a code.  I noticed other folks who repeated their correction up to 53 times, I'm not doing that.

I find out some fields are named with a D instead of a B or a G instead of a C, so I introduce another suffix.  Also for B and C fields, there are only 20 clones by field instead of 23 for A.  I end up with 3 variables to manage :


and the 23 ifs statements are replaced by 5 those lines :


For block of code specific to the first iteration, I just need to add exceptions such as :


Finally, the code ends up with 814 lines

After this work, I can investigate the code and understand something, instead of wasting my time.  2-3 hours later, I could explain it to the Business Analyst.

jeudi 25 septembre 2014

Classes

Introduction 

My first course of programming (C++) in college introduced me to the concept of classes and objects; I had to use OO programming in most of IT courses in the 3 years before I god my degree. Having been working in various environments (Web, Databases, C++ Programming) before jumping into Peoplesoft, I didn't practice OO Programming in my early years, or considered working with classes and objects as a constraint of the programming language. This was also my approach when I jumped into the Peoplesoft world, until...

When I started working as a Peoplesoft Developer, I started modifying pages here and there, then I jumped in SQR and Peoplecode, discovered the APIs, and eventually the functions. I remember that, for a reuse purpose, I created a generic function to generate a CSV file with as input the file name and path, the separator, the quotes, etc... I reused my function to generate other CSV files and it worked great. One day, I got to develop the opposite : a CSV file reader. I tried to follow the same logic, but the treatment of each CSV line changes from a case to another; I tried to find a way to call dynamically a function without having to hardcode the call in a switch-case, or using @, such as when we get a record from its name contained in a string variable. I didn't find anything until...

Until... I read about Application Packages. The solution was to create a CSVReader class with an abstract method ReadLine (called for each line read in ReadFile) which shall be implemented in each subclass - the real CSV Readers. Ok I might be to advanced for beginners.

Example

I think the best way to explain the concept of classes is to give an example with commonly used classes. Everyone should know the Record class, from the APIs; it is used at large in a lot of Peoplecode pieces. You don't know how the Record type is implemented, and you don't really want to. However, you can declare a variable of type Record, you can interact with it using some functions (GetField, Insert, SelectByKeyEffDt ), and you can visualize its properties (FieldCount, Name). You can create different variables of type Record, built differently (using CreateRecord(), GetRecord(), ...); you can build a record from scratch, insert manipulates rows, or you can populate it from the component processor.

The functions (methods) and properties that you can use to manipulate an object of type Record (available to you) are the public elements of the class.  Basically, everything you see in the APIs Peoplebook consist of the public properties and methods.

If there is a public part in a class, there should be a private part; of course, and there is even a protected part, but we can forget it for now.  The private part consists of all the properties and functions that you don't want the programmers to be able to manipulate.  Let's imagine that the way Oracle implemented the Record class.  A possible implementation (not the best) would be to have two variables, an array of string for the field names, and an array of array of any for the values (1 array per line).  What happens if a programmer decides to insert a variable in one of the arrays ?  The whole organization might become messed-up.  Another programmer might decide to add a field in the list of field names; same result.  One of the goal of having private elements is to avoid those problems; the core elements should only be modified following precise rules, rules that you implement in the public methods.  Of course, sometimes, it can be ok to let the user modify a variable directly; some purists will tell you that you need a getter and a setter (methods that returns the value or that sets the value), but this is not the most important at my eyes.

To be continued...

Resources

Application Packages - Oracle Peoplebooks
Object Oriented Programming - Wikipedia

mercredi 10 septembre 2014

Example of refactorization

To understand Peoplecode is one thing, to develop correctly using Peopletools is an other game.

I know sometimes it is easier to copy-paste multiple time the same piece of code, or to duplicate 10 times the same fields in a record, I did that in the past, when starting coding.  However, it is not the best practice at my eyes : the code is hard to maintain, because it is very long, but also because when a correction must be done, it must be applied for each copy of the code.

In my example, a table was built to store student insurance data for each semester; basically the structure of the table looks like :

EMPLID
STRM
EFFDT
EFFSEQ
FIELD1_SEP
FIELD2_SEP
FIELD3_SEP
...
FIELD1_AUG
FIELD2_AUG
FIELD3_AUG

First Problem : To repeat 12 times the same 3 fields in the same record is not appropriate, especially in this case, where some fields were used for a semester only (Ex : for Fall : SEP, OCT, NOV, DEC ).

To solve this problem, I created a child table : 

EMPLID
STRM
EFFDT
EFFSEQ
EFFSEQ2
MONTH
FIELD1
FIELD2
FIELD3


Now, lets jump to the code (second problem).  In fact, this was the first problem in time, I solve it, after, when I realized I couldn't work with the former record, I created the new child record and replace completely the code.

So, the code I had to read, decode in my head, understand, and modify, was about 800 lines long.  It was an Application Engine Peoplecode.  I started looking at the code, and quickly understood that it was made of the same block copied and pasted a lot of times, with minor differences; actually, I found out that there were two different blocks, repeated respectively 12 and 32 times.

Block1 For all months from SEP to currmonth - 1 -> 32 times
      If STATERECORD_AET.FIELD1_NOV <> "X" Then
         If STATERECORD_AET.UM_MONTH_NOV = "Y" Then
            If STATERECORD_AET.FIELD3_NOV <> 0 Then
               STATERECORD_AET.FIELD2_NOV = ( - STATERECORD_AET.FIELD3_NOV);
            Else
               STATERECORD_AET.FIELD2_NOV = ( - &MTTRI);
            End-If;
         Else
            STATERECORD_AET.FIELD2_NOV = &MTTRI;
         End-If;        
         STATERECORD_AET.FIELD1_NOV = "X";
      End-If;

Block2 1 time for each month -> 12 times       If STATERECORD_AET.FIELD1_DEC <> "X" Then
         If STATERECORD_AET.UM_MONTH_DEC = "Y" Then
            STATERECORD_AET.FIELD2_DEC = ( - STATERECORD_AET.FIELD3_DEC) + (&MTTRI);
         Else
            STATERECORD_AET.FIELD2_DEC = &MTTRI;
         End-If;
         STATERECORD_AET.FIELD1_DEC = "X";
      End-If;

The code was organized like this :

IF strm = fall THEN
   IF month = 04 (December) THEN
        Part1 for September
        Part1 for October
        Part1 for November
        Part2 for December
  END-IF;
   IF month = 03 (November) THEN
        Part1 for September
        Part1 for October
        Part2 for November
  END-IF;

And so on, for each semester, for each month.  I can't imagine that someone copied the same code 32 times and replaced the suffix of each field... any way.

There was no way I would repeate a single change 12, 32 or 44 times.  So I took few hours to see if I could replace all this copy-past by the two parts only.  In fact, the only difference between the same part copied for two different months was the suffix corresponding to the month.  I remember that we can use the @ character to get an object (a field, a record) from a string containing its name, such as :

&rec1.GetField(@("FIELD." | &fieldname));

So I copied each block ONLY 1 MORE TIME, replaced the field name by the dynamic way to retrieve it, and applied the logic to apply each part the good number of times for each month.

Here is the result :

/*The months suffixes ; one field had suffixes in French and others had suffixes in English*/
&FieldSuffixes = CreateArray("SEP", "OCT", "NOV", "DEC", "JAN", "FEB", "MAR", "APR", "MAY", "JUN", "JUL", "AUG");
&AMTSuffixes = CreateArray("SEP", "OCT", "NOV", "DEC", "JAN", "FEV", "MAR", "AVR", "MAY", "JUN", "JUL", "AUG");

&MonthNumber = 0;


/*BLOCK1 being repeated for each month before the current month*/

For &MonthNumber = 1 To Value(APPENGINE_AET.CURRMONTH.Value) - 1
   &currFieldSuffix = &FieldSuffixes.Get(&MonthNumber);
   &currAmtFieldSuffix = &AMTSuffixes.Get(&MonthNumber);
   
   &currMthReported = &rec1.GetField(@("FIELD.FIELD1_" | &currFieldSuffix)).Value;
   &currMthMonth = &rec1.GetField(@("FIELD.FIELD2_" | &currFieldSuffix)).Value;
   &currMthAmount = &rec1.GetField(@("FIELD.FIELD3_" | &currFieldSuffix)).Value;
   
   /*BLOCK1*/
   If &currMthReported <> "X" Then
      If &currMthMonth = "Y" Then
         If &existsLowerRow = "X" Then 
            If &currMthAmount <> 0 Then
               &rec1.GetField(@("FIELD.FIELD3_" | &currAmtFieldSuffix )).Value = ( - &currMthAmount);
            Else
               &rec1.GetField(@("FIELD.FIELD3_" | &currAmtFieldSuffix )).Value = ( - &MTTRI);
            End-If;
         End-If; 
         
      Else
         &rec1.GetField(@("FIELD.FIELD3_" | &currAmtFieldSuffix )).Value = &MTTRI;
      End-If;
      
      &rec1.GetField(@("FIELD.FIELD1_" | &currFieldSuffix)).Value = "X";
   End-If;
   
   
End-For;

&currFieldSuffix = &FieldSuffixes.Get(Value(APPENGINE_AET.CURRMONTH.Value));

&currAmtFieldSuffix = &AMTSuffixes.Get(Value(APPENGINE_AET.CURRMONTH.Value));

&currMthReported = &rec1.GetField(@("FIELD.FIELD1_" | &currFieldSuffix)).Value;

&currMthMonth = &rec1.GetField(@("FIELD.FIELD2_" | &currFieldSuffix)).Value;
&currMthAmount = &rec1.GetField(@("FIELD.FIELD3_" | &currFieldSuffix)).Value;

/*BLOCK2*/

If &currMthReported <> "X" Then
   If &currMthMonth = "Y" Then
      &rec1.GetField(@("FIELD.FIELD3_" | &currAmtFieldSuffix)).Value = ( - &currMthAmount) + (&MTTRI);
      
   Else
      &rec1.GetField(@("FIELD.FIELD3_" | &currAmtFieldSuffix)).Value = &MTTRI;
   End-If;
   &rec1.GetField(@("FIELD.FIELD1_" | &currFieldSuffix)).Value = "X";
   
End-If;


I know this is not rocket science, but not everybody would have thought to use dynamic fields.  Some strong developper debugged this code before me but none of them had this idea.  This is why I share it, so next time you see an opportunity to reduce the code, you may be inspire and you know were to look for an example. 

mardi 9 septembre 2014

Purpose of this blog

As of today, September 9th, 2014, I decided to create this blog to share my knowledge of Peoplesoft Development.

I have been working for more than 10 years since I graduated at the University of Sherbrooke (QC, Canada). I worked few years in different environments (DBA, Web Development, PLM) before jumping into the marvellous world of Peoplesoft. I firstly worked two years for a university before I started working as a consultant; I was assigned to different clients in the banking, education, IT & retail industries. Some were more organized than others, had a better understanding and proficiency with Peoplesoft and Peopletools, but I could always find weaknesses and bad practices.

Peoplebooks and other Peoplesoft documentation tools are a great source of information, but sometimes, they are not very helpful. I experimented this unpleasant experience when having to configure Integration Broker between two PS Environments, under pressure and without knowledge. With some good help from Oracle Support, I managed to make everything work, before I started to understand.

You might already know Jim Marion & Hakan Biroglu, who maintain two marvellous blogs, which are great source of information; by the way, Hakan Biroglu shared a great 80 pages PDF document on Integration Broker, which would have been very useful when I was under pressure ( http://hakanbiroglu.blogspot.ca/2013/01/integration-broker-basics-for.html#.Uz2qTPl5OT8 ). My intention is not to concurrence them, I know I wouldn't reach their ankle (see my documentation on IB : http://bjpsft.blogspot.ca/2012/04/my-integration-broker-experience.html ). I just want to share some of my work, some of my ideas, to show the world other possibilities in the Peoplesoft world.

 Julien