Steven Wang’s Blog – All about data: dig, distill, visualize, prognosticate & prescribe

Categories

All (12)

AI practice & execution (1)

Analytics leadership (1)

ChatGPT (1)

Cohen’s Kappa (1)

ConSumer Health (1)

Confusion Matrix (1)

Data Governance (2)

Data Process Framework (1)

Data Quality (2)

Data Science (3)

Data Solution Architect (1)

Data Visuliazation (3)

Data fundamentals (1)

Density Distribution (1)

Explorative Data Analysis (1)

FTP Folder Upload (1)

Fleiss’ Kappa (1)

Julia Programming (1)

Jupyter Notebook (1)

LLMs (1)

Machine Learning (1)

Master Data Management (1)

Mathematics (1)

Performance Mesaures (1)

Personal Development (1)

Prompt Engineering (1)

Python (1)

Quarto (1)

R Shiny (1)

RMarkdown (1)

ROS & SPPD (1)

Random Forest (1)

Reproducible report (1)

Segmentation (1)

Team Management (1)

Window Function (1)

When Demos Fly but Projects Walk: An AI Fitness Guide

AI practice & execution

Data fundamentals

Data Governance

Data Quality

Analytics leadership

AI gives you wings, but you’ll spend most days walking. This cheeky guide shows why skipping “leg day” (fundamentals, BS detection, judgment) turns flashy demos into faceplants. Learn to keep your craft strong, let AI handle the grunt work, and still walk home when the cloud catches fire.

ADPR: Adaptive Data Process Refinement Architecture

A Forward-thinking and Practical Framework for Modern Data Processing

Data Process Framework

Data Solution Architect

Data Governance

Data Quality

Master Data Management

The Adaptive Data Process Refinement (ADPR) architecture offers a practical framework for modern data challenges through four sequential stages: Acquired, DeCoded, Purified, and Released. This quality-first approach emphasizes master data standardization, comprehensive quality assurance, and strategic value delivery with embedded governance. ADPR addresses common data processing issues—quality inconsistencies, integration complexity, and scalability limitations—while providing the transparency and adaptability organizations need for reliable decision-making and competitive advantage.

Carving a Mark: A Decade of Dedication on My 10-Year Working Anniversary

Personal Development

Team Management

Embark on a journey of self-reflection as I commemorate a decade of dedication and mark my 10-year working anniversary in the AU/NZ healthcare industry. From the thrill of overcoming challenges to the joy of achieving milestones, I have passionately driven myself to make a meaningful impact in data science.

The Nitty-Gritty of Retail Product Velocity and ROS/SPPD Metrics

ConSumer Health

Performance Mesaures

ROS & SPPD

This article aims to explain the concept of product velocity and its importance in the retail industry. Product velocity is a metric that measures the rate of sales of a product within a specific store and is crucial for monitoring sales performance and creating effective sales strategies. The article discusses the interplay between velocity and distribution, and how retailers rely on velocity as a principal metric when allocating limited shelf space. It also explains how product velocity can determine a product’s success or failure in the retail industry. The article concludes by introducing two key product velocity measures used by business analysts: Rate of Sales (ROS) and Sales per Point of Distribution (SPPD).

ChatGPT Prompt Optimization Guide

ChatGPT

Prompt Engineering

LLMs

Optimizing the prompt input is important because it helps ensure that the model produces accurate and relevant responses. A well-optimized prompt includes relevant keywords and phrases that the model can use to generate the most appropriate response.

Using Python `ftppretty` and `pathlib` Libraries to upload a Nested Local Folder to an FTP Site

Python

FTP Folder Upload

Normally, an FTP client app can only upload files under one folder on the FTP site at a time. This is extremely cumbersome! This utility is used to upload an entire nested hierarchical local directory to an FTP site in one go.

I’m `Quartoed` - Converted to using Quarto for my writings and my new blog!!

Quarto

Reproducible report

RMarkdown

Jupyter Notebook

After many years of using R Markdown and Python Jupyter notebook, I am now converted to Quarto, the new generation of scientific and technical writing tool.

Fun with Complex Quadratic Polynomial Julia Set by Using Julia Programing

Julia Programming

Mathematics

Data Visuliazation

Data Science

A Julia set is an area boundary formed by points in the complex number plane or the Riemann sphere

Inter Rater Reliability Study with Cohen’s Kappa and Fleiss’ Kappa

Cohen’s Kappa

Fleiss’ Kappa

Confusion Matrix

Density Distribution

Data Visuliazation

Data Science

This report is a study on the inter-rater reliability analysis based on the manual annotation of 80 different articles by 81 different raters who participated this online rating exercise.

Using Shiny Server to Deploy Interactive Visualization Part 1: Data Exploratory Analysis of Red Wine Data

R Shiny

Explorative Data Analysis

Data Visuliazation

Data Science

This Red Wine Quality data exploratory app demonstrates a core feature of Shiny: interactivity and reactivity.

A Simple Way of Quantiling Process in T-SQL by using Window Function and Ceiling Function

Window Function

Segmentation

Quantiling process is very powerful when 2 dimensional of sales data is available, like product sales and market sales. In business it is very common to use quantile matrix to perform the account segmentation.

Prediction of Correctness on Barbell Lifting Exercise Based on Accelerometer Data

Machine Learning

Random Forest

In this project, I will use data from accelerometers on the belt, forearm, arm, and dumbell to predict which class a participant was performing.