Continuous Integration (CI) is an overloaded term meaning many things to different people. When it comes to the purely technical aspects, CI is often defined as the process of automatically building and testing your application’s code each time a developer pushes a commit to a remote source-code control repository like GitLab or GitHub.
The use of CI is an integral part of DevOps and can be used to great effect: lead times (e.g. the time it takes from idea to shipping) can be reduced, and thanks to automated unit and integration testing, error rates typically drop as well.
In most cases a so-called CI server coordinates the execution of tests using one or more CI Pipelines. CI Pipelines are typically defined as code, and checked into the same Git repository as the application. Most CI servers accept YAML syntax to define jobs and associate them to stages such as linting, build, test, deploy.
Before pushing changes to the remote repository developers typically perform local testing first.
Local Tests
Testing code locally before committing/pushing it to the remote Git repository is very important to avoid problems with the CI Pipeline’s execution. This article concerns itself with local tests of JavaScript modules used with Oracle Database 23ai on Linux x86-64 or aarch64.
The ability to maintain JavaScript modules as schema objects are an interesting new database feature allowing developers to process data where they live using one of the most popular programming languages. With the addition of JavaScript Oracle Database 23ai features 3 programming languages (PL/SQL and Java being the other 2). The feature enabling JavaScript support on Linux x86-64 in Oracle Database 23ai is known as Multilingual Engine (MLE).
Testing your code typically involves the following steps:
- checking for syntactical correctness
- linting
- unit testing
- others (including security!)
The easiest way to generate good code right from the beginning is to care for these at the time you write the code. Since JavaScript doesn’t know about types it might be difficult to detect wrong arguments passed to an API. Therefore the choice fell to Typescript. This should allow you to detect problems with your code quickly, however it comes at the expense of a little extra work. This extra work however is almost guaranteed to pay off.
The MLE team has provided the TypeScript declarations used by all MLE built-in modules on NPM, documentation can be found on GitHub.
A previous version of this blog post used typescript-eslint for linting, however due to the deprecation of inflight it might no longer be safe to use. A replacement was found in Biome, which seems to be pretty popular in the React community.
NOTE: The use of biome is merely an example, not an endorsement for the tool. There are many more linters available to developers. Always ensure you are comfortable working with a given piece of software, and that you are (license, …) compliant.
Versions Used
The following versions were used in the creation of this article:
$ cat package.json
{
"devDependencies": {
"@biomejs/biome": "1.9.4",
"mle-js": "^23.6.0"
},
"dependencies": {
"typescript": "^5.7.3"
}
}
I picked Node 20 LTS as the runtime on Linux x86-64.
Initial JavaScript Example
For the sake of demonstration let’s assume someone created the following MLE JavaScript module in their favourite editor, saved as src/badfile.js. It’s deliberately riddled with issues, don’t use it other than a bad example on how not to do things.
/**
* Update the "lastUpdated" field in a purchase order, adding it if it does not
* yet exist. Uses the example defined in the JSON Developer's Guide, chapter 4
* Examples 4-1 and 4-3
*
* @param {object} purchaseOrder - the PO to update
* @param {string} lastUpdate - a string representation of a date (YYYY-MM-DDThh:mm:ss)
* @returns {object} the updated purchaseOrder
*/
function setLastUpdatedDate(purchaseOrder, lastUpdate) {
if (purchaseOrder === undefined) {
throw Error("unknown purchase order");
}
if (lastUpdate = undefined) {
lastUpdate = new Date().toISOString();
}
console.log(`last update set to ${lastUpdate}`);
purchaseOrder.lastUpdate = lastUpdate;
return purchaseOrder;
}
/**
* Use vanilla JavaScript to validate a PurchaseOrder. This could have been
* done with JSON schema validation as well
*
* @param {object} purchaseOrder - the PO to validate
* @returns {boolean} true if the PO could be successfully validated, false if not
*/
function validatePO(purchaseOrder) {
// a PO must contain line items
if (purchaseOrder.LineItems.length <= 0) {
return false;
}
// a PO must contain shipping instructions
if (purchaseOrder.ShippingInstructions === undefined) {
return false;
}
return true;
}
/**
* Fetch a PurchaseOrder from the database and process it. Store the last modification
* timestamp alongside
*
* @param {number} poNumber - the PONumber as stored in j_purchaseorder.po_document.PONumber
*/
export function processPurchaseOrder(poNumber) {
const result = session.execute(
`SELECT
po.po_document as PO
FROM
j_purchaseorder po
WHERE
po.po_document.ponumber = :1`,
[poNumber],
" thisIsAnIncorrectParameter "
);
// ensure the PO exists
if (result.rows === undefined) {
throw Error(`could not find a PO for PO Number ${poNumber}`);
} else {
const myPO = result.rows[0].PO;
}
// make sure the PO is valid
if (!validatePO(myPO)) {
throw Error(`Purchase Order ${poNumber} is not a valid PO`);
}
// do some fancy processing with the PO
// indicate when the last operation happened
myPO = setLastUpdatedDate(myPO, undefined);
result = session.execute(
`UPDATE j_purchaseorder po
SET
po.po_document = :myPO
WHERE
po.po_document.PONumber = :poNumber`,
{
myPO: {
dir: oracledb.BIND_IN,
type: oracledb.DB_TYPE_JSON,
val: myPO
},
poNumber: {
dir: oracledb.BIND_IN,
type: oracledb.NUMBER,
val: poNumber
}
}
);
if (result.rowsAffected != 1) {
throw Error(`unable to persist purchase order ${poNumber}`);
}
}
Before submitting the code to the database for testing the developer should ensure the code doesn’t have any errors. The example has been chosen to put emphasis on the fact that it’s very hard to ensure code quality by merely eyeballing the text ;)
Improving Code Quality
Converting the above code to TypeScript provides lots of benefits to developers:
- Type declarations available for all MLE modules add a safety net to the code.
- Linting TypeScript adds even more checks to ensure the transpiled code is good to go.
The remainder of this article is concerned with the transition to TypeScript and correction of the errors encountered.
Installing MLE JavaScript Type Declarations
As per the MLE module documentation the first step is to install the mle-js module from NPM. This module contains the type declarations for all built-in modules used later. The following instructions are relative to the project root directory.
npm install mle-js --save-dev
Installing the Linter
In the next step you need to decide which linter to use, this post focuses on biome as per the introduction.
npm install --save-dev --save-exact @biomejs/biome
The first step working with biome is to create a configuration. The following code snippet shows the configuration used for this post. I created it using npx @biomejs/biome init, it’s a sane default and good starting point. Adjust as needed for your project.
{
"$schema": "https://biomejs.dev/schemas/1.9.4/schema.json",
"vcs": {
"enabled": false,
"clientKind": "git",
"useIgnoreFile": false
},
"files": {
"ignoreUnknown": false,
"ignore": []
},
"formatter": {
"enabled": true,
"indentStyle": "tab"
},
"organizeImports": {
"enabled": true
},
"linter": {
"enabled": true,
"rules": {
"recommended": true
}
},
"javascript": {
"formatter": {
"quoteStyle": "double"
}
}
}
JavaScript to TypeScript
The first step in the JavaScript to Typescript conversion is to rename the source file to src/goodFile.ts. Next you need to create a tsconfig.json file, needed for type-checking. The following is a minimal file instructing TypeScript to transpile JavaScript code with the latest version of JavaScript supported by Oracle Database 23ai (at the time of writing):
{
"compilerOptions": {
"target": "ESNext",
"module": "ESNext",
"rootDir": "src",
"outDir": "dist",
"noEmitOnError": true,
"esModuleInterop": true,
"forceConsistentCasingInFileNames": true,
"strict": true,
"skipLibCheck": false,
"lib": [
"ES6",
"ES2017",
"ES2021"
]
}
}
You also need to install TypeScript in your project, this is done using the now familiar npm i --save-dev typescript command.
TypeScript code must be told about the MLE JavaScript type definitions found in mle-js. This is done using a triple-slash directive right at the start of the document.
/// <reference types="mle-js" />
/**
* Update the "lastUpdated" field in a purchase order, adding it if it does not
* yet exist. Uses the example defined in the JSON Developer's Guide, chapter 4
* Examples 4-1 and 4-3
* @param {object} purchaseOrder - the PO to update
* @param {string} lastUpdate - a string representation of a date (YYYY-MM-DDThh:mm:ss)
* @returns {object} the updated purchaseOrder
*/
function setLastUpdatedDate(purchaseOrder, lastUpdate) {
/* more code */
You will almost immediately see things light up in red in your IDE. If not, you can use the command line to check for problems. Let’s start with the compilation:
$ npx tsc --pretty false
src/goodFile.ts(12,29): error TS7006: Parameter 'purchaseOrder' implicitly has an 'any' type.
src/goodFile.ts(12,44): error TS7006: Parameter 'lastUpdate' implicitly has an 'any' type.
src/goodFile.ts(36,21): error TS7006: Parameter 'purchaseOrder' implicitly has an 'any' type.
src/goodFile.ts(57,38): error TS7006: Parameter 'poNumber' implicitly has an 'any' type.
src/goodFile.ts(67,9): error TS2559: Type '" thisIsAnIncorrectParameter "' has no properties in common with type 'IExecuteOptions'.
src/goodFile.ts(78,21): error TS2304: Cannot find name 'myPO'.
src/goodFile.ts(85,5): error TS2304: Cannot find name 'myPO'.
src/goodFile.ts(85,31): error TS2304: Cannot find name 'myPO'.
src/goodFile.ts(87,5): error TS2588: Cannot assign to 'result' because it is a constant.
src/goodFile.ts(97,22): error TS2304: Cannot find name 'myPO'.
The first step is to add types to get rid of the `any` type errors. These are stubs, but they hopefully show the added value of adding types to the module:
/// <reference types="mle-js" />
/**
* Stub interfaces, for demonstration purpose only, should be fleshed out to
* reflect all data as per Example 4-3 in chapter 4 of the JSON Developer's
* Guide
*/
interface ILineItem {
ItemNumber: number,
Part: string,
Quantity: number
}
interface IShippingInstructions {
name: string,
address: string,
phone: string
}
interface IPurchaseOrder {
PONumber: number,
lastUpdate: string,
LineItems: ILineItem[],
ShippingInstructions: IShippingInstructions[]
}
/**
* Update the "lastUpdated" field in a purchase order, adding it if it does not
* yet exist. Uses the example defined in the JSON Developer's Guide, chapter 4
* Examples 4-1 and 4-3
*
* @param {object} purchaseOrder - the PO to update
* @param {string} lastUpdate - a string representation of a date (YYYY-MM-DDThh:mm:ss)
* @returns {object} the updated purchaseOrder
*/
function setLastUpdatedDate(purchaseOrder: IPurchaseOrder, lastUpdate: string): IPurchaseOrder {
if (purchaseOrder === undefined) {
/* ... more code ... */
After all the types have been defined and referred to in the code, the next pass looks a lot better, and it now catches quite a few problems:
$ npx tsc --pretty false
src/goodFile.ts(41,9): error TS2322: Type 'undefined' is not assignable to type 'string'.
src/goodFile.ts(89,9): error TS2559: Type '" thisIsAnIncorrectParameter "' has no properties in common with type 'IExecuteOptions'.
src/goodFile.ts(100,21): error TS2304: Cannot find name 'myPO'.
src/goodFile.ts(107,5): error TS2304: Cannot find name 'myPO'.
src/goodFile.ts(107,31): error TS2304: Cannot find name 'myPO'.
src/goodFile.ts(107,37): error TS2345: Argument of type 'undefined' is not assignable to parameter of type 'string'.
src/goodFile.ts(109,5): error TS2588: Cannot assign to 'result' because it is a constant.
src/goodFile.ts(119,22): error TS2304: Cannot find name 'myPO'.
Quite a few of them are common mistakes, like trying to assign a value to a constant after it has been initialised. Other errors include the infamous assignment vs comparison operator (line 41) and so on.
One of the prime benefits of TypeScript – type checking – is clearly demonstrated in line 89. In this line the code tries to pass an unknown option to the `execute()` statement. With pure JavaScript this wouldn’t be detected until the first execution (hopefully in a unit test).
Let’s fix these errors as well. The resulting file is shown here for reference. Please note that the types are merely stubs to drive the point home, a proper implementation would add all the other required fields from the PO as well.
/// <reference types="mle-js" />
/**
* Stub interfaces, for demostration purpose only, should be fleshed out to
* reflect all data as per Example 4-3 in chapter 4 of the JSON Developer's
* Guide
*/
interface ILineItem {
ItemNumber: number,
Part: string,
Quantity: number
}
interface IShippingInstructions {
name: string,
address: string,
phone: string
}
interface IPurchaseOrder {
PONumber: number,
lastUpdate: string,
LineItems: ILineItem[],
ShippingInstructions: IShippingInstructions[]
}
/**
* Update the "lastUpdated" field in a purchase order, adding it if it does not
* yet exist. Uses the example defined in the JSON Developer's Guide, chapter 4
* Examples 4-1 and 4-3
* @param {object} purchaseOrder - the PO to update
* @param {string} lastUpdate - a string representation of a date (YYYY-MM-DDThh:mm:ss)
* @returns {object} the updated purchaseOrder
*/
function setLastUpdatedDate(purchaseOrder: IPurchaseOrder, lastUpdate: string): IPurchaseOrder {
if (purchaseOrder === undefined) {
throw Error("unknown purchase order");
}
if (lastUpdate === undefined) {
lastUpdate = new Date().toISOString();
}
console.log(`last update set to ${lastUpdate}`);
purchaseOrder.lastUpdate = lastUpdate;
return purchaseOrder;
}
/**
* Use vanilla JavaScript to validate a PurchaseOrder. This could have been
* done with JSON schema validation as well
* @param {object} purchaseOrder - the PO to validate
* @returns {boolean} true if the PO could be successfully validated, false if not
*/
function validatePO(purchaseOrder: IPurchaseOrder): boolean {
// a PO must contain line items
if (purchaseOrder.LineItems.length <= 0) {
return false;
}
// a PO must contain shipping instructions
if (purchaseOrder.ShippingInstructions === undefined) {
return false;
}
return true;
}
/**
* Fetch a PurchaseOrder from the database and process it. Store the last modification
* timestamp alongside
* @param {number} poNumber - the PONumber as stored in j_purchaseorder.po_document.PONumber
*/
export function processPurchaseOrder(poNumber: IPurchaseOrder["PONumber"]): void {
let result = session.execute(
`SELECT
po.po_document as PO
FROM
j_purchaseorder po
WHERE
po.po_document.PONumber = :1`,
[ poNumber ]
);
// ensure the PO exists
if (result.rows === undefined || result.rows.length === 0) {
throw new Error(`could not find Purchase Order ${poNumber}`);
}
let myPO = result.rows[0].PO as IPurchaseOrder;
// make sure the PO is valid
if (! validatePO(myPO)) {
throw new Error(`Purchase Order ${poNumber} failed validation`);
}
// do some (imaginary) fancy processing with the PO ...
// ... then: indicate when the last operation happened
myPO = setLastUpdatedDate(myPO, "");
result = session.execute(
`UPDATE j_purchaseorder po
SET
po.po_document = :myPO
WHERE
po.po_document.PONumber = :poNumber`,
{
myPO: {
dir: oracledb.BIND_IN,
type: oracledb.DB_TYPE_JSON,
val: myPO
},
poNumber: {
dir: oracledb.BIND_IN,
type: oracledb.DB_TYPE_NUMBER,
val: poNumber
}
}
);
if (result.rowsAffected = 1) {
throw new Error(`unable to persist purchase order ${poNumber}`);
}
}
At this stage, the TypeScript transpiler stops complaining:
$ npx tsc --pretty false && echo 'well done!'
well done!
Next up: Linting
The above code looks all right on first inspection, doesn’t it? Well, you might be in for a surprise. The linter has uncovered some interesting details:
$ npx biome lint --error-on-warnings --colors=off ./src
./src/goodFile.ts:41:20 lint/suspicious/noDoubleEquals FIXABLE ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
× Use === instead of ==
39 │ }
40 │
> 41 │ if (lastUpdate == undefined) {
│ ^^
42 │ lastUpdate = new Date().toISOString();
43 │ }
i == is only allowed when comparing against null
39 │ }
40 │
> 41 │ if (lastUpdate == undefined) {
│ ^^
42 │ lastUpdate = new Date().toISOString();
43 │ }
i Using == may be unsafe if you are relying on type coercion
i Unsafe fix: Use ===
41 │ ····if·(lastUpdate·===·undefined)·{
│ +
./src/goodFile.ts:127:29 lint/suspicious/noDoubleEquals FIXABLE ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
× Use !== instead of !=
125 │ );
126 │
> 127 │ if (result.rowsAffected != 1) {
│ ^^
128 │ throw new Error(`unable to persist purchase order ${poNumber}`);
129 │ }
i != is only allowed when comparing against null
125 │ );
126 │
> 127 │ if (result.rowsAffected != 1) {
│ ^^
128 │ throw new Error(`unable to persist purchase order ${poNumber}`);
129 │ }
i Using != may be unsafe if you are relying on type coercion
i Unsafe fix: Use !==
127 │ ····if·(result.rowsAffected·!==·1)·{
│ +
./src/goodFile.ts:42:9 lint/style/noParameterAssign ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
× Reassigning a function parameter is confusing.
41 │ if (lastUpdate == undefined) {
> 42 │ lastUpdate = new Date().toISOString();
│ ^^^^^^^^^^
43 │ }
44 │
i The parameter is declared here:
33 │ * @returns {object} the updated purchaseOrder
34 │ */
> 35 │ function setLastUpdatedDate(purchaseOrder: IPurchaseOrder, lastUpdate: string): IPurchaseOrder {
│ ^^^^^^^^^^^^^^^^^^
36 │
37 │ if (purchaseOrder === undefined) {
i Use a local variable instead.
Checked 2 files in 1667µs. No fixes applied.
Found 3 errors.
lint ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
× Some errors were emitted while running checks.
Well, that’s quite a list! Time to fix the code some more. Since it’s a bit tedious switching from command line to editor, many IDEs offer support for linting. Here’s an example of the same file in VSCode:
Using IDE support for linting and syntax checking is the best way moving forward. If your list of problems is zero, chances are high that your code will pass the CI Pipeline, too.
But surely you can’t be serious!?!
Every linter comes with an opinionated set of rules, but you or your team doesn’t necessarily share these opinion. In the above example, biome complained about me redefining a parameter passed to the function:
./src/goodFile.ts:42:9 lint/style/noParameterAssign ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
× Reassigning a function parameter is confusing.
41 │ if (lastUpdate == undefined) {
> 42 │ lastUpdate = new Date().toISOString();
│ ^^^^^^^^^^
43 │ }
44 │
i The parameter is declared here:
33 │ * @returns {object} the updated purchaseOrder
34 │ */
> 35 │ function setLastUpdatedDate(purchaseOrder: IPurchaseOrder, lastUpdate: string): IPurchaseOrder {
│ ^^^^^^^^^^^^^^^^^^
36 │
37 │ if (purchaseOrder === undefined) {
i Use a local variable instead.
At this stage you can either tell the linter to ignore this rule:
/* ... */
if (lastUpdate === undefined) {
// biome-ignore lint/style/noParameterAssign: this is a predefined variable in MLE
lastUpdate = new Date().toISOString();
}
console.log(`last update set to ${lastUpdate}`);
purchaseOrder.lastUpdate = lastUpdate;
/* ... */
Alternatively, fix the error. Which is probably just as easy:
/* ... */
purchaseOrder.lastUpdate =
lastUpdate === undefined ? new Date().toISOString() : lastUpdate;
/* ... */
Don’t forget to add linting and syntax checking to your CI Pipeline
With the hard part completed you can now add your linter and formatter to your toolchain. Git Hooks are pretty good for this, but more importantly, add support for linting, syntax checking and your unit tests into your CI Pipeline.
Summary
This has been a rather long read, but it hopefully showed how to use MLE type-declarations to make your life as a developer easier.
Happy coding! The full example is available at my GitHub.
Martins Blog 