The IDX Market Intelligence Dashboard is a Python-based market analysis project that combines three different IDX datasets into a single analytical workflow. The project focuses on Market Movers, Top Stocks, and Smart Money Flow for BBCA.

The system retrieves data through the IDX API using RapidAPI, processes different JSON response structures, cleans and identifies relevant analytical fields, and presents the results through a three-panel dashboard.

The workflow consists of five cells covering API configuration, data collection, adaptive JSON normalization, analytical processing, and final dashboard visualization. The objective is to transform raw market data into a practical overview of stock performance, transaction value, and smart money activity.

Cell 1 — Setup, API Key, and Helper Function

The first cell prepares the Python environment, configures RapidAPI authentication, and creates a reusable API request function.

# -*- coding: utf-8 -*-

# =====================================================
# IDX SMART BROKER INTELLIGENCE PROJECT
# =====================================================

import requests
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
from getpass import getpass

# =========================
# RAPIDAPI CONFIGURATION
# =========================
RAPIDAPI_HOST = "indonesia-stock-exchange-idx.p.rapidapi.com"
RAPIDAPI_KEY = getpass("Masukkan RapidAPI Key: ")

HEADERS = {
    "Content-Type": "application/json",
    "x-rapidapi-host": RAPIDAPI_HOST,
    "x-rapidapi-key": RAPIDAPI_KEY
}

BASE_URL = f"https://{RAPIDAPI_HOST}"

def get_api(endpoint, params=None):
    """Mengambil data API dengan error handling."""
    try:
        response = requests.get(
            BASE_URL + endpoint,
            headers=HEADERS,
            params=params,
            timeout=30
        )
        response.raise_for_status()
        return response.json()

    except requests.exceptions.RequestException as e:
        print(f"API Request Error: {e}")
        return {}

    except ValueError:
        print("Response API bukan JSON.")
        return {}

Explanation

This cell establishes the API connection and creates get_api() as a reusable function for retrieving IDX data. The API key is entered through getpass, preventing it from being directly displayed in the notebook. The function also handles request and JSON errors.

Cell 2 — Retrieve Three IDX Datasets

The second cell collects the three datasets used by the dashboard: top gainers, top stocks by transaction value, and BBCA smart money flow.

# ==========================================================
# 1. MARKET MOVER - TOP GAINER
# ==========================================================
market_mover_raw = get_api(
    "/api/movers/top-gainer",
    params={
        "filterStocks":
        "FILTER_STOCKS_TYPE_MAIN_BOARD,FILTER_STOCKS_TYPE_DEVELOPMENT_BOARD"
    }
)

# ==========================================================
# 2. TOP STOCKS
# ==========================================================
top_stocks_raw = get_api(
    "/api/market-detector/top-stock",
    params={
        "start": "2026-01-05",
        "end": "2026-01-05",
        "valueType": "VALUE_TYPE_TOTAL",
        "investorType": "INVESTOR_TYPE_ALL",
        "marketType": "MARKET_TYPE_ALL",
        "page": 1
    }
)

# ==========================================================
# 3. SMART MONEY FLOW - BBCA
# ==========================================================
smart_money_raw = get_api(
    "/api/analysis/bandar/smart-money/BBCA",
    params={"days": 30}
)

print("Market Mover :", "OK" if market_mover_raw else "FAILED")
print("Top Stocks   :", "OK" if top_stocks_raw else "FAILED")
print("Smart Money  :", "OK" if smart_money_raw else "FAILED")

Explanation

The project retrieves three complementary datasets. Market Mover identifies top gainers, Top Stocks provides transaction-value information, and Smart Money retrieves 30 days of BBCA smart money data. The system then reports whether each API request was successful.

Cell 3 — Adaptive JSON Parser

Different API endpoints may return different JSON structures. This cell therefore uses an endpoint-aware normalization process to locate relevant records and convert them into pandas DataFrames.

# ============================================================
# CELL 3 - ENDPOINT-AWARE JSON NORMALIZATION
# ============================================================

import pandas as pd
import numpy as np
import json


def find_record_list(data, preferred_keys=None):
    """
    Mencari list-of-dict dengan memprioritaskan key
    yang relevan untuk endpoint tertentu.
    """

    preferred_keys = preferred_keys or []

    if isinstance(data, list):
        if data and all(isinstance(x, dict) for x in data):
            return data
        return []

    if not isinstance(data, dict):
        return []

    # 1. Prioritas berdasarkan key endpoint
    for key in preferred_keys:

        for real_key, value in data.items():

            if str(real_key).lower() == key.lower():

                if isinstance(value, list):

                    if value and all(
                        isinstance(x, dict)
                        for x in value
                    ):
                        return value

                elif isinstance(value, dict):

                    result = find_record_list(
                        value,
                        preferred_keys
                    )

                    if result:
                        return result

    # 2. Recursive fallback
    for value in data.values():

        if isinstance(value, (dict, list)):

            result = find_record_list(
                value,
                preferred_keys
            )

            if result:
                return result

    return []


def normalize_endpoint(data, preferred_keys):

    records = find_record_list(
        data,
        preferred_keys
    )

    if records:

        return pd.json_normalize(
            records,
            sep="_"
        )

    # response dictionary tunggal
    if isinstance(data, dict):

        # unwrap "data"
        inner = data.get("data", data)

        if isinstance(inner, dict):

            try:
                return pd.json_normalize(
                    [inner],
                    sep="_"
                )
            except:
                pass

    return pd.DataFrame()


# ============================================================
# MARKET MOVER
# ============================================================

df_movers = normalize_endpoint(
    market_mover_raw,
    [
        "gainers",
        "topGainers",
        "stocks",
        "items",
        "results",
        "content",
        "list",
        "data"
    ]
)


# ============================================================
# TOP STOCKS
# ============================================================

df_top = normalize_endpoint(
    top_stocks_raw,
    [
        "stocks",
        "topStocks",
        "items",
        "list",
        "results",
        "content",
        "data"
    ]
)


# ============================================================
# SMART MONEY
# ============================================================

smart_container = (
    smart_money_raw.get("data", smart_money_raw)
    if isinstance(smart_money_raw, dict)
    else smart_money_raw
)

df_smart = normalize_endpoint(
    smart_container,
    [
        "smartMoney",
        "smart_money",
        "history",
        "chart",
        "items",
        "list",
        "results",
        "content",
        "data"
    ]
)


print("=" * 75)
print("NORMALIZED IDX API DATA")
print("=" * 75)

print("\nMARKET MOVER")
print("Shape   :", df_movers.shape)
print("Columns :", list(df_movers.columns))

print("\nTOP STOCKS")
print("Shape   :", df_top.shape)
print("Columns :", list(df_top.columns))

print("\nSMART MONEY BBCA")
print("Shape   :", df_smart.shape)
print("Columns :", list(df_smart.columns))

Explanation

The adaptive parser searches for relevant list structures based on preferred endpoint keys. If the expected structure is not found, it recursively searches nested dictionaries and lists. The normalized outputs are stored as df_movers, df_top, and df_smart.

Cell 4 — Data Inspection and Market Summary

This cell cleans numerical data, detects relevant columns automatically, calculates fallback percentage changes when necessary, and identifies useful smart money variables.

# ============================================================
# CELL 4 - CLEANING & ANALYTICAL ENGINE
# ============================================================

import re


def find_column(df, candidates):

    if df.empty:
        return None

    cols = list(df.columns)

    mapping = {
        str(c).lower(): c
        for c in cols
    }

    # Exact match
    for candidate in candidates:

        key = candidate.lower()

        if key in mapping:
            return mapping[key]

    # Partial match
    for candidate in candidates:

        key = candidate.lower()

        for lower_col, original in mapping.items():

            if key in lower_col:
                return original

    return None


def clean_numeric(series):

    if series is None:
        return pd.Series(dtype=float)

    s = series.astype(str)

    s = (
        s.str.replace("%", "", regex=False)
         .str.replace(",", "", regex=False)
         .str.replace(
             r"[^0-9eE+\-.]",
             "",
             regex=True
         )
    )

    return pd.to_numeric(
        s,
        errors="coerce"
    )


# ============================================================
# MARKET MOVER COLUMNS
# ============================================================

mover_symbol = find_column(
    df_movers,
    [
        "symbol",
        "ticker",
        "stockCode",
        "stock_code",
        "code"
    ]
)

mover_price = find_column(
    df_movers,
    [
        "price",
        "lastPrice",
        "last_price",
        "close",
        "last"
    ]
)

mover_change = find_column(
    df_movers,
    [
        "priceChange",
        "price_change",
        "change",
        "gain"
    ]
)

mover_percent = find_column(
    df_movers,
    [
        "changePercent",
        "change_percent",
        "percentage",
        "percent",
        "gainPercent",
        "gain_percent"
    ]
)


# ============================================================
# CLEAN MARKET MOVER
# ============================================================

if mover_price:

    df_movers["_price"] = clean_numeric(
        df_movers[mover_price]
    )

if mover_change:

    df_movers["_change"] = clean_numeric(
        df_movers[mover_change]
    )

if mover_percent:

    df_movers["_percent"] = clean_numeric(
        df_movers[mover_percent]
    )

else:

    df_movers["_percent"] = np.nan


# ============================================================
# FALLBACK PERCENTAGE CALCULATION
# ============================================================

# current_price = previous_price + change
#
# previous_price = current_price - change
#
# percent_change =
# change / previous_price * 100
# ============================================================

if (
    "_price" in df_movers.columns
    and "_change" in df_movers.columns
):

    previous_price = (
        df_movers["_price"]
        - df_movers["_change"]
    )

    calculated_percent = np.where(
        previous_price > 0,
        (
            df_movers["_change"]
            / previous_price
        ) * 100,
        np.nan
    )

    # Pakai hasil kalkulasi bila percentage API
    # kosong atau semuanya bernilai nol.
    percent_valid = (
        df_movers["_percent"]
        .replace([np.inf, -np.inf], np.nan)
        .dropna()
    )

    if (
        percent_valid.empty
        or percent_valid.abs().sum() == 0
    ):

        df_movers["_percent"] = (
            calculated_percent
        )


# ============================================================
# TOP STOCKS
# ============================================================

top_symbol = find_column(
    df_top,
    [
        "symbol",
        "ticker",
        "stockCode",
        "stock_code",
        "code"
    ]
)

top_value = find_column(
    df_top,
    [
        "totalValue",
        "total_value",
        "transactionValue",
        "transaction_value",
        "value"
    ]
)

if top_value:

    df_top["_value"] = clean_numeric(
        df_top[top_value]
    )


# ============================================================
# SMART MONEY
# ============================================================

# Konversi semua kolom yang memungkinkan ke numeric
smart_numeric = pd.DataFrame(
    index=df_smart.index
)

for col in df_smart.columns:

    converted = clean_numeric(
        df_smart[col]
    )

    # hanya masukkan kalau memang ada numeric valid
    if converted.notna().sum() > 0:

        # Hindari date/timestamp/id sebagai flow
        name = str(col).lower()

        excluded = [
            "date",
            "time",
            "year",
            "month",
            "timestamp",
            "id"
        ]

        if not any(x in name for x in excluded):

            smart_numeric[col] = converted


# ============================================================
# PRIORITIZE FLOW-LIKE COLUMNS
# ============================================================

flow_keywords = [
    "flow",
    "net",
    "foreign",
    "bandar",
    "accum",
    "distrib",
    "buy",
    "sell",
    "money"
]


smart_flow_columns = [
    col
    for col in smart_numeric.columns
    if any(
        key in str(col).lower()
        for key in flow_keywords
    )
]


if smart_flow_columns:

    smart_analysis = (
        smart_numeric[
            smart_flow_columns
        ].copy()
    )

else:

    smart_analysis = (
        smart_numeric.copy()
    )


print("=" * 75)
print("DETECTED ANALYTICAL FIELDS")
print("=" * 75)

print(
    "Market Mover :",
    mover_symbol,
    "| Price:",
    mover_price,
    "| Change:",
    mover_change,
    "| Percent:",
    mover_percent
)

print(
    "Top Stocks   :",
    top_symbol,
    "| Value:",
    top_value
)

print(
    "Smart Money numeric columns:",
    list(smart_analysis.columns)
)


print("\nMARKET MOVER SAMPLE")

cols_show = [
    x for x in [
        mover_symbol,
        mover_price,
        mover_change
    ]
    if x is not None
]

display(
    df_movers[
        cols_show + ["_percent"]
    ].head(10)
)


print("\nSMART MONEY SAMPLE")

display(
    smart_analysis.head(10)
)

Explanation

The analytical engine automatically detects relevant columns instead of depending on one fixed API schema. Numerical values are cleaned before analysis, while missing percentage data can be calculated from price and price-change values.

For Smart Money data, the system converts possible numerical fields and prioritizes variables containing terms such as flow, net, foreign, bandar, accum, distrib, buy, sell, and money.

Cell 5 — Final IDX Market Intelligence Dashboard

The final cell combines the processed datasets into a three-panel visualization covering market gainers, transaction-value leaders, and BBCA smart money activity. It also generates a textual market summary.

# ============================================================
# CELL 5 - FINAL IDX MARKET INTELLIGENCE DASHBOARD
# ============================================================

import matplotlib.pyplot as plt


fig, axes = plt.subplots(
    1,
    3,
    figsize=(20, 6)
)


# ============================================================
# PANEL 1 - MARKET MOVERS
# ============================================================

if (
    mover_symbol
    and "_percent" in df_movers.columns
):

    movers_plot = (
        df_movers[
            [
                mover_symbol,
                "_percent"
            ]
        ]
        .dropna()
        .sort_values(
            "_percent",
            ascending=False
        )
        .head(10)
    )

    if not movers_plot.empty:

        axes[0].bar(
            movers_plot[
                mover_symbol
            ].astype(str),
            movers_plot[
                "_percent"
            ]
        )

        axes[0].set_title(
            "Top 10 Market Gainers"
        )

        axes[0].set_ylabel(
            "Price Change (%)"
        )

        axes[0].tick_params(
            axis="x",
            rotation=45
        )

    else:

        axes[0].text(
            0.5,
            0.5,
            "No valid gainer data",
            ha="center",
            va="center"
        )

else:

    axes[0].text(
        0.5,
        0.5,
        "Market Mover unavailable",
        ha="center",
        va="center"
    )


# ============================================================
# PANEL 2 - TOP STOCKS
# ============================================================

if (
    top_symbol
    and "_value" in df_top.columns
):

    top_plot = (
        df_top[
            [
                top_symbol,
                "_value"
            ]
        ]
        .dropna()
        .sort_values(
            "_value",
            ascending=False
        )
        .head(10)
    )

    axes[1].bar(
        top_plot[
            top_symbol
        ].astype(str),
        top_plot[
            "_value"
        ] / 1e12
    )

    axes[1].set_title(
        "Top Stocks by Transaction Value"
    )

    axes[1].set_ylabel(
        "Transaction Value (Trillion IDR)"
    )

    axes[1].tick_params(
        axis="x",
        rotation=45
    )

else:

    axes[1].text(
        0.5,
        0.5,
        "Top Stock data unavailable",
        ha="center",
        va="center"
    )


# ============================================================
# PANEL 3 - SMART MONEY
# ============================================================

smart_status = "NO DATA"
smart_net = np.nan


if not smart_analysis.empty:

    # ========================================================
    # CASE A:
    # MULTIPLE ROWS -> cari numeric variable paling informatif
    # ========================================================

    if len(smart_analysis) > 1:

        variability = (
            smart_analysis
            .std(numeric_only=True)
            .sort_values(
                ascending=False
            )
        )

        valid_cols = (
            variability[
                variability > 0
            ].index.tolist()
        )

        if valid_cols:

            flow_col = valid_cols[0]

            values = (
                smart_analysis[
                    flow_col
                ]
                .dropna()
                .reset_index(drop=True)
            )

            axes[2].plot(
                range(1, len(values) + 1),
                values,
                marker="o"
            )

            axes[2].axhline(
                0,
                linewidth=1
            )

            axes[2].set_title(
                f"BBCA Smart Money - {flow_col}"
            )

            axes[2].set_xlabel(
                "Observation"
            )

            axes[2].set_ylabel(
                "Flow Value"
            )

            smart_net = values.sum()

            if smart_net > 0:

                smart_status = (
                    "POSITIVE NET FLOW"
                )

            elif smart_net < 0:

                smart_status = (
                    "NEGATIVE NET FLOW"
                )

            else:

                smart_status = "NEUTRAL"

        else:

            axes[2].text(
                0.5,
                0.5,
                "Smart Money numeric data\nhas no variation",
                ha="center",
                va="center"
            )


    # ========================================================
    # CASE B:
    # SINGLE ROW -> tampilkan metric values
    # ========================================================

    else:

        row = (
            smart_analysis
            .iloc[0]
            .dropna()
        )

        row = row[
            row.abs() > 0
        ]

        if not row.empty:

            row = row.iloc[:8]

            axes[2].bar(
                range(len(row)),
                row.values
            )

            axes[2].set_xticks(
                range(len(row))
            )

            axes[2].set_xticklabels(
                [
                    str(x)[:14]
                    for x in row.index
                ],
                rotation=45,
                ha="right"
            )

            axes[2].set_title(
                "BBCA Smart Money Metrics"
            )

            smart_net = row.sum()

            smart_status = (
                "METRICS AVAILABLE"
            )

        else:

            axes[2].text(
                0.5,
                0.5,
                "No usable Smart Money metrics",
                ha="center",
                va="center"
            )

else:

    axes[2].text(
        0.5,
        0.5,
        "Smart Money data unavailable",
        ha="center",
        va="center"
    )


plt.suptitle(
    "INDONESIA STOCK EXCHANGE - MARKET INTELLIGENCE DASHBOARD",
    fontsize=16,
    fontweight="bold"
)

plt.tight_layout()
plt.show()


# ============================================================
# FINAL MARKET SUMMARY
# ============================================================

print("\n" + "=" * 75)
print("MARKET INTELLIGENCE SUMMARY")
print("=" * 75)


# ------------------------------------------------------------
# TOP GAINER
# ------------------------------------------------------------

if (
    mover_symbol
    and "_percent" in df_movers.columns
):

    valid = df_movers.dropna(
        subset=["_percent"]
    )

    if not valid.empty:

        winner = valid.loc[
            valid["_percent"].idxmax()
        ]

        print(
            f"Top Gainer    : "
            f"{winner[mover_symbol]} "
            f"({winner['_percent']:.2f}%)"
        )


# ------------------------------------------------------------
# MARKET LEADER
# ------------------------------------------------------------

if (
    top_symbol
    and "_value" in df_top.columns
):

    valid = df_top.dropna(
        subset=["_value"]
    )

    if not valid.empty:

        leader = valid.loc[
            valid["_value"].idxmax()
        ]

        print(
            f"Market Leader : "
            f"{leader[top_symbol]} | "
            f"Rp {leader['_value']:,.0f}"
        )


# ------------------------------------------------------------
# SMART MONEY
# ------------------------------------------------------------

print(
    f"BBCA Smart Money : "
    f"{smart_status}"
)

if pd.notna(smart_net):

    print(
        f"Smart Metric Net : "
        f"{smart_net:,.2f}"
    )

print("=" * 75)

Explanation

The final dashboard contains three analytical panels. The first displays the top 10 market gainers, the second ranks stocks by transaction value, and the third analyzes available BBCA Smart Money metrics. The Smart Money panel can process either multiple observations or a single-row metric response.

The dashboard also generates a final textual summary identifying the top gainer, market leader by transaction value, and BBCA Smart Money status.

Understanding the Dashboard

The dashboard combines three different market perspectives.

Market Movers identifies stocks with the strongest positive price changes in the returned dataset. This provides a direct view of current market performance.

Top Stocks ranks stocks according to transaction value. High transaction value indicates substantial trading activity within the selected dataset, but it should not automatically be interpreted as buying pressure.

Smart Money Flow focuses on BBCA and evaluates available numerical smart money variables. When multiple observations are available, the system selects the variable with the highest variation as the primary flow series and calculates its aggregate value.

Together, these components provide a broader market overview than relying on a single indicator.

Limitations

The dashboard depends on the structure and availability of IDX API responses. The adaptive parser improves flexibility, but changes in API fields or response structures can still affect the analysis.

Smart money, broker, and transaction-value indicators should not be treated as standalone trading signals. They provide market context and should be evaluated together with price action, company fundamentals, valuation, liquidity, market conditions, and risk management.

Conclusion

The IDX Market Intelligence Dashboard demonstrates how Python can integrate IDX market data into a structured analytical workflow. The project retrieves Market Movers, Top Stocks, and BBCA Smart Money Flow data before processing and visualizing the results.

The five-cell architecture separates API configuration, data collection, JSON normalization, analytical processing, and dashboard visualization. This structure makes the project practical for further development and market research.

The resulting dashboard provides a concise view of market gainers, transaction-value leaders, and smart money activity. It should be used as an analytical framework rather than an independent trading decision system.