home > research > 2019-software-development-radar (2024-10-30)

Core Technologies Overview in 2019: JavaScript, Python, Scheme, and Clojure

Core Technologies Overview in 2019: JavaScript, Python, Scheme, and Clojure

Table of Contents

Background

This consists of an overview of some of the core technologies available in 2019.

Most of the examples here will be a combination of JavaScript, Python, Scheme, and Clojure.

Some of the core elements of https://www.thoughtworks.com/radar/techniques will be included in the review specifically as it relates to ML.

  • CD4ML
  • productionizing Jupyter Notebooks

Compiler

AST: Esprima

http://esprima.org/ 

const esprima = require('esprima');
const program = 'const answer = 42';
esprima.tokenize(program);

AST: CSS

REPL

Task: Incrementally build a lisp interpreter capable of running the following: 

(define a 1)

(define b (lambda (x) (+ 1 x)))

(b 1)

(+ 1 1)

(quote 1)

(car (quote (1 2)))

(cdr (quote (1 2)))

(cond (true) 1 2)

(define fib
  (lambda (n)
        (if (or (= n 1)
                        (= n 2))
                  n
                  (+ (fib (- n 1))
                         (fib (- n 2))))))

(fib 5)

Interpreters

https://github.com/kanaka/mal/blob/master/process/guide.md

Evaluation

Data Structures

Array

Node

Linked Lists

Dictionary

BST

Heapsort

Graphs

Types

Design

Swagger

Provide some background on OpenAPI v3 particularly as it relates to paths, requests, and responses.

SOLID

  • Single responsibility principle
  • Open/closed principle
  • Liskov substitution principle
  • Interface segregation principle
  • Dependency inversion principle

https://en.wikipedia.org/wiki/SOLID_(object-oriented_design)

SRS

  • The Single Responsibility Principle

There should never be more than one reason for a class to change.

http://www.objectmentor.com/resources/articles/srp.pdf

Chaining

Immutability

Security

OWASP

There should be some familiarity with mechanism for designing secure systems that take the OWASP top ten into account.

As part of design there should be concepts of standard request and response objects, schema parsing, field validation, default values, and error handling.

Injection

CORS

Indicate how to enable CORS on a server.

https://enable-cors.org/server_nginx.html 

  server = http.createServer(function(req,res){
          res.setHeader('Access-Control-Allow-Origin', '*');
          res.setHeader('Access-Control-Request-Method', '*');
          res.setHeader('Access-Control-Allow-Methods', 'OPTIONS, GET');
          res.setHeader('Access-Control-Allow-Headers', 'authorization, content-type');
          if ( req.method === 'OPTIONS' ) {
                  res.writeHead(200);
                  res.end();
                  return;
          }
  });

JWT

See the encoding example noted in https://github.com/jwalsh/jwt-example/blob/master/index.js or generate a token from https://jwt.io/ . 

  const encoded = 'eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiaWF0IjoxNTE2MjM5MDIyfQ.SflKxwRJSMeKKF2QT4fwpMeJf36POk6yJV_adQssw5c';

  const base64UrlEncode = (string) => {
          return btoa(encodeURIComponent(string));
  };

  const base64UrlDencode = (string) => {
          return decodeURIComponent(atob(string));
  };

  console.log(base64UrlDencode(base64UrlEncode(JSON.stringify([1, 2, 3]))))

  const decode = (encoded) => {
          const [h, p, s] = encoded.split('.');
          return base64UrlDencode(p);
  }

  console.log(JSON.parse(decode(encoded)))

Static analysis

SOX

https://www.sarbanes-oxley-101.com/sarbanes-oxley-checklist.htm

  • Establish safeguards to prevent data tampering
  • Establish safeguards to establish timelines
  • Establish verifiable controls to track data access
  • Ensure that safeguards are operational
  • Periodically report the effectiveness of safeguards
  • Detect Security Breaches
  • Disclose security safeguards to SOX auditors
  • Disclose security breaches to SOX auditors
  • Disclose failures of security safeguards to SOX auditors

Risk assessment

DREAD

https://en.wikipedia.org/wiki/DREAD_(risk_assessment_model)

  • Damage – how bad would an attack be?
  • Reproducibility – how easy is it to reproduce the attack?
  • Exploitability – how much work is it to launch the attack?
  • Affected users – how many people will be impacted?
  • Discoverability – how easy is it to discover the threat?

STRIDE

  • Spoofing of user identity
  • Tampering
  • Repudiation
  • Information disclosure (privacy breach or data leak)
  • Denial of service (D.o.S)
  • Elevation of privilege

SSL

server=dev.wal.sh
openssl req -out $server.csr -new -newkey rsa:4096 -sha256 -nodes -keyout $server.key -subj "/C=US/ST=Massachusetts/L=Boston/O=Walsh/OU=IT Department/CN=$server"

Data

GraphQL

UI

CSS Preprocessors

  • SASS
  • LESS
  • Garden

CSS Styles

Coding

Python: core

Python: numpy

Python: sklearn

JavaScript: core

JavaScript: web

JavaScript: React

JavaScript: lodash

TypeScript: core

Java: core

  • Enumeration
  • BitSet
  • Vector
  • Stack
  • Dictionary
  • Hashtable
  • Properties

Clojure: core

Clojure: core/async

ClojureScript: core

Rust

Go

Ruby: Rails

DataViz

Statistics

stochastic

https://github.com/jwalsh/stochastic 

var stochastic = require('@jwalsh/stochastic');
const exp = stochastic.exp(20);

Machine Learning

Linear regression

x,y
-26,-52
-279,-558
-163,-326
-254,-508
72,144
-120,-240
401,802
-297,-594
238,476


import matplotlib.pyplot as plot
import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LinearRegression

Features and models

{
  "version" : "1.0",
  "rowId" : null,
  "rowWeight" : null,
  "targetAttributeName" : "y",
  "dataFormat" : "CSV",
  "dataFileContainsHeader" : true,
  "attributes" : [ {
    "attributeName" : "x",
    "attributeType" : "NUMERIC"
  }, {
    "attributeName" : "y",
    "attributeType" : "NUMERIC"
  } ],
  "excludedAttributeNames" : [ ]
}

Schemas

Resource

Estimation

Interpretibility

  • What-If Tool - model comparison

Tooling

Eslint

Prettier

Emacs

Postman

Testing

pytest

Jest

AWS

https://docs.aws.amazon.com/index.html

Overview

  • Region > Availability zone
  • Compute, sstorate, db, tools, security, crypto, ml, scaling, migration, mobile, networking, analytics, integrations, IoT, VR, SDKs

Ansible

CLI

touch .tmp
aws s3 cp .tmp s3://$USER-tmp/

Environments

Ansible

Kubernetes

Services

Maven

Artifactory

GitHub

GitLab

Docker

docker run \
  --rm \
  -u root \
  -p 8080:8080 \
  -v jenkins-data:/var/jenkins_home \
  -v /var/run/docker.sock:/var/run/docker.sock \
  -v "$HOME":/home \
  jenkinsci/blueocean

Ansible

Slack

Jira

DataDog

Kafka

  • Broker
  • Consumer
  • Group
  • Message
  • Offset
  • Partition
  • Pipeline
  • Producer
  • Stream
  • Topic

Vault

Secrets as a service.

Deployment

CI/CD

Operations

  • unix services

Logging

https://docs.google.com/presentation/d/14PiMsvqs7f5A3_leDE07HoQkbHf_sUfuNHDShUMyT3A/edit#slide=id.g731d2a787_0_13

  • Crash
  • Enter debugger
  • Throw exception
  • Return special value
  • Set error flag
  • Recover, keep going

Debugging

Defects

  • Defect: An incorrect program code (a bug in the code).
  • Infection: An incorrect program state (a bug in the state).
  • Failure: An observable incorrect program behavior (a bug in the behavior).

Why Programs Fail: A Guide to Systematic Debugging, 2ed.

TRAFFIC

  • Track the problem in the database.
  • Reproduce the failure.
  • Automate and simplify the test case.
  • Find possible infection origins.
  • Focus on the most likely origins:
    • Code smells
    • Known infections
    • Causes in state, code, and input
  • Isolate the cause
  • Correct the defect.

http://www.whyprogramsfail.com/slides.php

Types

Pairs

const pair = (_first, _rest) => (_) => _(_first, _rest);
const first = (_pair) => _pair((_first, _rest) => _first);
const rest = (_pair) => _pair((_first, _rest) => _rest);

Refactoring

  • Duplicate Observed Data
  • Encapsulate Downcast
  • Encapsulate Field
  • Extract Interface
  • Introduce Foreign Method
  • Introduce Local Extension
  • Pull Up Constructor Body
  • Pull Up Field
  • Push Down Field
  • Replace Delegation with Inheritance
  • Replace Type Code with Class
  • Replace Type Code with Subclasses
  • Add Parameter
  • Change Bidirectional Association to Unidirectional
  • Change Reference to Value
  • Change Unidirectional Association to Bidirectional
  • Change Value to Reference
  • Collapse Hierarchy
  • Consolidate Conditional Expression
  • Consolidate Duplicate Conditional Fragments
  • Decompose Conditional
  • Dynamic Method Definition
  • Eagerly Initialized Attribute
  • Encapsulate Collection
  • Extract Class
  • Extract Method
  • Extract Module
  • Extract Subclass
  • Extract Superclass
  • Extract Surrounding Method
  • Extract Variable
  • Form Template Method
  • Hide Delegate
  • Hide Method
  • Inline Class
  • Inline Method
  • Inline Module
  • Inline Temp
  • Introduce Assertion
  • Introduce Class Annotation
  • Introduce Expression Builder
  • Introduce Gateway
  • Introduce Named Parameter
  • Introduce Null Object
  • Introduce Parameter Object
  • Isolate Dynamic Receptor
  • Lazily Initialized Attribute
  • Move Eval from Runtime to Parse Time
  • Move Field
  • Move Method
  • Parameterize Method
  • Preserve Whole Object
  • Pull Up Method
  • Push Down Method
  • Recompose Conditional
  • Remove Assignments to Parameters
  • Remove Control Flag
  • Remove Middle Man
  • Remove Named Parameter
  • Remove Parameter
  • Remove Setting Method
  • Remove Unused Default Parameter
  • Rename Method
  • Replace Abstract Superclass with Module
  • Replace Array with Object
  • Replace Conditional with Polymorphism
  • Replace Constructor with Factory Method
  • Replace Data Value with Object
  • Replace Delegation With Hierarchy
  • Replace Dynamic Receptor with Dynamic Method Definition
  • Replace Error Code with Exception
  • Replace Exception with Test
  • Replace Hash with Object
  • Replace Inheritance with Delegation
  • Replace Loop with Collection Closure Method
  • Replace Magic Number with Symbolic Constant
  • Replace Method with Method Object
  • Replace Nested Conditional with Guard Clauses
  • Replace Parameter with Explicit Methods
  • Replace Parameter with Method
  • Replace Record with Data Class
  • Replace Subclass with Fields
  • Replace Temp with Chain
  • Replace Temp with Query
  • Replace Type Code with Module Extension
  • Replace Type Code With Polymorphism
  • Replace Type Code with State/Strategy
  • Self Encapsulate Field
  • Separate Query from Modifier
  • Split Temporary Variable
  • Substitute Algorithm

SDLC

Agile

Estimation

Metrics

Four key metrics

  • lead time,
  • deployment frequency,
  • mean time to restore (MTTR)
  • change fail percentage

Projects

See some of the external repositories for the project work.

Make a Lisp

Data Structures

Re-implement the core APIs for stacks, enumerations, lists, etc.

React+Apollo

https://www.typescriptlang.org/docs/handbook/react-&-webpack.html 

  cd ~/sandbox
  if [ ! -d apollo-example ]
  then
          nvm use 10
          npx create-react-app apollo-example
  fi
  cd apollo-example
  mkdir -p src/components

 npm i parcel-bundler -D
 npm i typescript -D
 npm i -D @types/react @types/react-dom

IT

LDAP

VPN

On-boarding/Off-boarding

Conferences

Training

Author: Jason Walsh

j@wal.sh

Last Updated: 2024-10-30 16:43:54