Dividend stocks often attract investors because they combine two important signals: corporate action and market interest.

A stock may appear attractive because it has an upcoming dividend schedule, but that does not always mean the stock is currently active in the market. On the other hand, a stock may be trending because of price movement, volume, or transaction value, but it may not have any dividend catalyst.

This notebook builds a practical case: creating an IDX dividend stock screener using Python.

The idea is simple. We combine dividend calendar data from getDividendCalendar with market attention data from getTrendingStocks. After both datasets are collected, the notebook cleans the data, merges them by ticker, calculates a combined score, and visualizes the top stocks.

The result is a screening workflow that helps identify trending stocks that also have dividend calendar information.

Cell 1 — Import Library + Config API

import os
import json
import time
from typing import Any, Dict, List, Optional, Union

import requests
import pandas as pd
import matplotlib.pyplot as plt


RAPIDAPI_HOST = "indonesia-stock-exchange-idx.p.rapidapi.com"
BASE_URL = f"https://{RAPIDAPI_HOST}"

# Ganti dengan RapidAPI Key Anda
RAPIDAPI_KEY = "ISI_RAPIDAPI_KEY_ANDA_DI_SINI"

# Alternatif jika memakai environment variable:
# RAPIDAPI_KEY = os.getenv("RAPIDAPI_KEY")

if not RAPIDAPI_KEY or RAPIDAPI_KEY == "ISI_RAPIDAPI_KEY_ANDA_DI_SINI":
    raise ValueError("Masukkan RapidAPI Key Anda dulu pada variabel RAPIDAPI_KEY.")

HEADERS = {
    "Content-Type": "application/json",
    "x-rapidapi-host": RAPIDAPI_HOST,
    "x-rapidapi-key": RAPIDAPI_KEY,
}

print("API configuration ready.")

Explanation

Cell 1 prepares the notebook environment and API configuration.

The notebook imports several libraries. os is prepared for environment variable usage. json is used to preview raw API responses. time is used to add delays between API calls. requests is used to call the API endpoints. pandas is used to process tabular data. matplotlib.pyplot is used for visualization.

The variable RAPIDAPI_HOST stores the RapidAPI host, while BASE_URL creates the base API URL.

The RAPIDAPI_KEY variable stores the API key. In this article, the real key is replaced with a placeholder for security.

The notebook also provides an alternative option using environment variables:

# RAPIDAPI_KEY = os.getenv("RAPIDAPI_KEY")

After that, the notebook validates whether the API key has been filled. If the key is empty or still uses the placeholder, the process stops with ValueError.

Finally, the HEADERS dictionary is created. These headers are used for every request sent to the API.

Cell 2 — Helper Function

def request_get(
    endpoint: str,
    params: Optional[Dict[str, Any]] = None,
    timeout: int = 30,
    max_retries: int = 5,
    wait_seconds: int = 3
) -> Union[Dict[str, Any], List[Any]]:

    url = f"{BASE_URL}{endpoint}"

    for attempt in range(1, max_retries + 1):
        time.sleep(wait_seconds)

        response = requests.get(
            url,
            headers=HEADERS,
            params=params,
            timeout=timeout
        )

        try:
            payload = response.json()
        except Exception:
            payload = response.text

        if response.ok:
            return payload

        if response.status_code == 429:
            print(
                f"Rate limit 429. Percobaan {attempt}/{max_retries}. "
                f"Tunggu {wait_seconds} detik..."
            )
            time.sleep(wait_seconds)
            continue

        raise RuntimeError(
            f"Request gagal. Status: {response.status_code}\n"
            f"URL: {response.url}\n"
            f"Response: {payload}"
        )

    raise RuntimeError(
        f"Request gagal karena rate limit 429 setelah {max_retries} percobaan."
    )


def find_first_list(obj: Any) -> List[Dict[str, Any]]:
    if isinstance(obj, list):
        return obj if all(isinstance(item, dict) for item in obj) else []

    if isinstance(obj, dict):
        priority_keys = [
            "data", "result", "results", "items", "rows",
            "content", "calendar", "dividend", "dividends",
            "stocks", "trending"
        ]

        for key in priority_keys:
            if key in obj:
                found = find_first_list(obj[key])
                if found:
                    return found

        for value in obj.values():
            found = find_first_list(value)
            if found:
                return found

    return []


def json_to_dataframe(payload: Union[Dict[str, Any], List[Any]]) -> pd.DataFrame:
    records = find_first_list(payload)

    if records:
        return pd.json_normalize(records)

    if isinstance(payload, dict):
        return pd.json_normalize(payload)

    return pd.DataFrame()


def detect_column(df: pd.DataFrame, keywords: List[str]) -> Optional[str]:
    if df.empty:
        return None

    lower_map = {col.lower(): col for col in df.columns}

    for keyword in keywords:
        if keyword.lower() in lower_map:
            return lower_map[keyword.lower()]

    for col in df.columns:
        col_lower = col.lower()
        if any(keyword.lower() in col_lower for keyword in keywords):
            return col

    return None


def safe_series(df: pd.DataFrame, col: Optional[str], default_value=None) -> pd.Series:
    if col is None or col not in df.columns:
        return pd.Series([default_value] * len(df), index=df.index)

    return df[col]


def normalize_0_100(series: pd.Series) -> pd.Series:
    numeric = pd.to_numeric(series, errors="coerce").fillna(0)

    min_val = numeric.min()
    max_val = numeric.max()

    if max_val == min_val:
        return pd.Series([0] * len(numeric), index=numeric.index)

    return (numeric - min_val) / (max_val - min_val) * 100

Explanation

Cell 2 contains helper functions used throughout the notebook.

The first function, request_get, handles GET requests to the API. It combines BASE_URL with the endpoint, waits for a few seconds, sends the request, and returns the payload if the response is successful.

This function also handles rate limits. If the API returns status code 429, the notebook waits and retries the request. This makes the workflow more stable when calling multiple endpoints.

The second function, find_first_list, searches through a nested JSON response and tries to find the first list of dictionaries. This is useful because API responses may store records inside different keys, such as data, calendar, dividend, stocks, or trending.

The third function, json_to_dataframe, converts the JSON payload into a pandas dataframe.

The fourth function, detect_column, detects a column based on possible keyword names. This is important because API responses may use different column names for ticker, price, volume, or dividend value.

The fifth function, safe_series, safely returns a dataframe column. If the column does not exist, it returns a default-value series.

The last function, normalize_0_100, converts numerical values into a 0–100 scale. This function is later used to create scoring columns.

Cell 3 — Ambil Data dari Dua API

print("Mengambil data getDividendCalendar...")
dividend_payload = request_get(
    endpoint="/api/calendar/dividend",
    wait_seconds=3
)

time.sleep(5)

print("Mengambil data getTrendingStocks...")
trending_payload = request_get(
    endpoint="/api/main/trending",
    wait_seconds=3
)


dividend_df = json_to_dataframe(dividend_payload)
trending_df = json_to_dataframe(trending_payload)

dividend_df["source_api"] = "getDividendCalendar"
trending_df["source_api"] = "getTrendingStocks"

print("Dividend Calendar shape:", dividend_df.shape)
print("Trending Stocks shape:", trending_df.shape)

display(dividend_df.head())
display(trending_df.head())

print("Preview Dividend Calendar JSON:")
print(json.dumps(dividend_payload, indent=2, ensure_ascii=False)[:1500])

print("\nPreview Trending Stocks JSON:")
print(json.dumps(trending_payload, indent=2, ensure_ascii=False)[:1500])

Explanation

Cell 3 retrieves data from the two APIs.

The first API call uses:

/api/calendar/dividend

This endpoint represents getDividendCalendar. It retrieves dividend calendar data from the IDX API.

After that, the notebook adds a manual delay using:

time.sleep(5)

This helps reduce the risk of hitting API rate limits.

The second API call uses:

/api/main/trending

This endpoint represents getTrendingStocks. It retrieves trending stock data.

Both API responses are converted into dataframes using json_to_dataframe.

The resulting dataframes are:

dividend_df

and

trending_df

The notebook then adds a source_api column to both datasets. This makes the data source easier to identify.

At the end, the notebook prints the shape of both dataframes, displays the first few rows, and prints a preview of the raw JSON response.

This cell is the main data collection stage for the IDX dividend stock screener.

Cell 4 — Cleaning Data Dividend dan Trending Stocks

# Deteksi kolom Dividend Calendar
dividend_ticker_col = detect_column(
    dividend_df,
    ["ticker", "symbol", "stockCode", "stock_code", "code", "securityCode"]
)

dividend_name_col = detect_column(
    dividend_df,
    ["name", "stockName", "stock_name", "companyName", "company_name", "issuer"]
)

cum_date_col = detect_column(
    dividend_df,
    ["cumDate", "cum_date", "cum", "cumDividendDate"]
)

ex_date_col = detect_column(
    dividend_df,
    ["exDate", "ex_date", "ex", "exDividendDate"]
)

recording_date_col = detect_column(
    dividend_df,
    ["recordingDate", "recording_date", "recordDate"]
)

payment_date_col = detect_column(
    dividend_df,
    ["paymentDate", "payment_date", "payDate", "distributionDate"]
)

dividend_value_col = detect_column(
    dividend_df,
    ["dividend", "dividendValue", "dividend_value", "cashDividend", "cash_dividend", "amount"]
)


# Deteksi kolom Trending Stocks
trending_ticker_col = detect_column(
    trending_df,
    ["ticker", "symbol", "stockCode", "stock_code", "code", "securityCode"]
)

trending_name_col = detect_column(
    trending_df,
    ["name", "stockName", "stock_name", "companyName", "company_name", "issuer"]
)

trending_price_col = detect_column(
    trending_df,
    ["price", "lastPrice", "last_price", "close", "currentPrice", "current_price"]
)

trending_change_col = detect_column(
    trending_df,
    ["change", "chg", "priceChange", "price_change", "point"]
)

trending_percent_col = detect_column(
    trending_df,
    ["percent", "percentage", "changePercent", "change_percent", "pct", "gain"]
)

trending_volume_col = detect_column(
    trending_df,
    ["volume", "vol", "transactionVolume", "transaction_volume"]
)

trending_value_col = detect_column(
    trending_df,
    ["value", "amount", "transactionValue", "transaction_value", "turnover"]
)


print("Detected Dividend columns:")
print("ticker         :", dividend_ticker_col)
print("name           :", dividend_name_col)
print("cum date       :", cum_date_col)
print("ex date        :", ex_date_col)
print("recording date :", recording_date_col)
print("payment date   :", payment_date_col)
print("dividend value :", dividend_value_col)

print("\nDetected Trending columns:")
print("ticker :", trending_ticker_col)
print("name   :", trending_name_col)
print("price  :", trending_price_col)
print("change :", trending_change_col)
print("percent:", trending_percent_col)
print("volume :", trending_volume_col)
print("value  :", trending_value_col)


dividend_clean = pd.DataFrame({
    "ticker": (
        safe_series(dividend_df, dividend_ticker_col, "")
        .astype(str)
        .str.upper()
        .str.strip()
    ),
    "stock_name_dividend": safe_series(dividend_df, dividend_name_col, ""),
    "cum_date": safe_series(dividend_df, cum_date_col, ""),
    "ex_date": safe_series(dividend_df, ex_date_col, ""),
    "recording_date": safe_series(dividend_df, recording_date_col, ""),
    "payment_date": safe_series(dividend_df, payment_date_col, ""),
    "dividend_value": pd.to_numeric(
        safe_series(dividend_df, dividend_value_col, None),
        errors="coerce"
    ),
})

dividend_clean = dividend_clean[dividend_clean["ticker"].ne("")].copy()
dividend_clean["source_api_dividend"] = "getDividendCalendar"


trending_clean = pd.DataFrame({
    "ticker": (
        safe_series(trending_df, trending_ticker_col, "")
        .astype(str)
        .str.upper()
        .str.strip()
    ),
    "stock_name_trending": safe_series(trending_df, trending_name_col, ""),
    "price": pd.to_numeric(
        safe_series(trending_df, trending_price_col, None),
        errors="coerce"
    ),
    "change": pd.to_numeric(
        safe_series(trending_df, trending_change_col, None),
        errors="coerce"
    ),
    "change_percent": pd.to_numeric(
        safe_series(trending_df, trending_percent_col, None),
        errors="coerce"
    ),
    "volume": pd.to_numeric(
        safe_series(trending_df, trending_volume_col, None),
        errors="coerce"
    ),
    "value": pd.to_numeric(
        safe_series(trending_df, trending_value_col, None),
        errors="coerce"
    ),
})

trending_clean = trending_clean[trending_clean["ticker"].ne("")].copy()
trending_clean["source_api_trending"] = "getTrendingStocks"


display(dividend_clean.head(20))
display(trending_clean.head(20))

Explanation

Cell 4 cleans and standardizes both datasets.

The first section detects important columns from dividend_df. The notebook searches for ticker, stock name, cum date, ex date, recording date, payment date, and dividend value columns.

The second section detects important columns from trending_df. The notebook searches for ticker, stock name, price, price change, percentage change, volume, and transaction value columns.

After detecting the columns, the notebook prints all detected column names. This helps users confirm whether the API response was interpreted correctly.

Next, the notebook creates dividend_clean. This dataframe standardizes dividend calendar data into:

ticker

stock_name_dividend

cum_date

ex_date

recording_date

payment_date

dividend_value

The ticker is converted into uppercase and stripped of extra spaces. The dividend value is converted into numeric format. Rows with empty tickers are removed.

Then, the notebook creates trending_clean. This dataframe standardizes trending stock data into:

ticker

stock_name_trending

price

change

change_percent

volume

value

Again, ticker values are cleaned and numerical columns are converted using pd.to_numeric.

This cell prepares both datasets so they can be merged by ticker in the next step.

Cell 5 — Gabungkan Data dan Buat Scoring

project_output = trending_clean.merge(
    dividend_clean,
    on="ticker",
    how="outer"
)

project_output["stock_name"] = project_output["stock_name_trending"].fillna(
    project_output["stock_name_dividend"]
)

project_output["change_percent_score"] = normalize_0_100(project_output["change_percent"])
project_output["volume_score"] = normalize_0_100(project_output["volume"])
project_output["value_score"] = normalize_0_100(project_output["value"])
project_output["dividend_score"] = normalize_0_100(project_output["dividend_value"])

project_output["has_dividend_calendar"] = project_output["source_api_dividend"].notna()
project_output["is_trending_stock"] = project_output["source_api_trending"].notna()

project_output["combined_score"] = (
    project_output["change_percent_score"] * 0.35
    + project_output["volume_score"] * 0.20
    + project_output["value_score"] * 0.20
    + project_output["dividend_score"] * 0.25
)

project_output["project_note"] = (
    "Data getDividendCalendar dan getTrendingStocks digabungkan berdasarkan ticker "
    "untuk melihat saham trending yang juga memiliki jadwal dividen."
)

project_output = project_output.sort_values("combined_score", ascending=False)

summary_df = pd.DataFrame({
    "description": [
        "Total saham dari getDividendCalendar",
        "Total saham dari getTrendingStocks",
        "Total saham setelah digabung",
        "Saham yang muncul di kedua API",
        "Saham trending yang punya jadwal dividen"
    ],
    "value": [
        len(dividend_clean),
        len(trending_clean),
        len(project_output),
        len(project_output[
            (project_output["has_dividend_calendar"] == True) &
            (project_output["is_trending_stock"] == True)
        ]),
        len(project_output[
            (project_output["has_dividend_calendar"] == True) &
            (project_output["is_trending_stock"] == True)
        ])
    ]
})

display(project_output.head(30))
display(summary_df)

Explanation

Cell 5 is the core scoring section of the IDX dividend stock screener.

The notebook merges trending_clean and dividend_clean using the ticker column. The merge uses how="outer", which means all tickers from both datasets are kept.

After merging, the notebook creates a unified stock name column called stock_name. It uses stock_name_trending first, and if that value is missing, it uses stock_name_dividend.

Then, the notebook creates four normalized scoring columns:

change_percent_score

volume_score

value_score

dividend_score

These scores are normalized into a 0–100 range using normalize_0_100.

The notebook then creates two boolean columns:

has_dividend_calendar

and

is_trending_stock

These columns indicate whether a stock appears in the dividend calendar dataset, the trending stock dataset, or both.

The final ranking metric is combined_score.

The formula gives:

35% weight to change_percent_score

20% weight to volume_score

20% weight to value_score

25% weight to dividend_score

This means the screener combines price momentum, trading activity, transaction value, and dividend signal into one score.

After the score is calculated, project_output is sorted from highest to lowest combined score.

The notebook also creates summary_df, which shows total stocks from each API, total stocks after merging, and the number of stocks that appear in both datasets.

This cell turns raw API data into a ranked dividend-based stock screener.

Cell 6 — Visualisasi dan Export

top_n = 15
plot_df = project_output.head(top_n).copy()

plt.figure(figsize=(10, 6))
plt.barh(plot_df["ticker"].astype(str), plot_df["combined_score"])
plt.gca().invert_yaxis()
plt.xlabel("Combined Score")
plt.ylabel("Ticker")
plt.title(f"Top {top_n} Trending Stocks + Dividend Calendar")
plt.tight_layout()
plt.show()


dividend_clean.to_csv("idx_dividend_calendar_clean.csv", index=False)
trending_clean.to_csv("idx_trending_stocks_clean.csv", index=False)
project_output.to_csv("idx_dividend_trending_project.csv", index=False)
summary_df.to_csv("idx_dividend_trending_summary.csv", index=False)

with pd.ExcelWriter("idx_dividend_trending_project.xlsx", engine="openpyxl") as writer:
    dividend_df.to_excel(writer, sheet_name="dividend_raw", index=False)
    trending_df.to_excel(writer, sheet_name="trending_raw", index=False)
    dividend_clean.to_excel(writer, sheet_name="dividend_clean", index=False)
    trending_clean.to_excel(writer, sheet_name="trending_clean", index=False)
    project_output.to_excel(writer, sheet_name="final_output", index=False)
    summary_df.to_excel(writer, sheet_name="summary", index=False)

print("Export selesai:")
print("- idx_dividend_calendar_clean.csv")
print("- idx_trending_stocks_clean.csv")
print("- idx_dividend_trending_project.csv")
print("- idx_dividend_trending_summary.csv")
print("- idx_dividend_trending_project.xlsx")

Explanation

Cell 6 visualizes and exports the final result.

The notebook sets:

top_n = 15

Then, it takes the top 15 rows from project_output and stores them in plot_df.

A horizontal bar chart is created using matplotlib. The Y-axis shows stock tickers, while the X-axis shows combined_score.

The chart title is:

Top 15 Trending Stocks + Dividend Calendar

This visualization helps users quickly identify stocks with the strongest combined score.

After visualization, the notebook exports several CSV files:

idx_dividend_calendar_clean.csv

idx_trending_stocks_clean.csv

idx_dividend_trending_project.csv

idx_dividend_trending_summary.csv

Then, the notebook creates one Excel file:

idx_dividend_trending_project.xlsx

The Excel file contains several sheets:

dividend_raw

trending_raw

dividend_clean

trending_clean

final_output

summary

This export process makes the final screening result easier to review, share, or continue in another analysis workflow.

Result:

Overall Workflow

This notebook follows a clear workflow.

First, it prepares the API configuration and required libraries.

Second, it creates helper functions for API requests, rate limit handling, JSON parsing, dataframe conversion, column detection, safe column selection, and score normalization.

Third, it retrieves dividend calendar data from getDividendCalendar.

Fourth, it retrieves trending stock data from getTrendingStocks.

Fifth, it cleans and standardizes both datasets.

Sixth, it merges both datasets by ticker.

Seventh, it calculates a combined_score based on momentum, volume, transaction value, and dividend value.

Finally, it visualizes the Top 15 stocks and exports the results into CSV and Excel files.

The output is an IDX dividend stock screener that helps identify trending stocks with dividend calendar signals.

Why This Case Matters

Many traders only look at trending stocks. Many dividend investors only look at dividend schedules. But combining both can create a more useful screening workflow.

A trending stock shows market attention. A dividend calendar shows corporate action context. When a stock appears in both datasets, it may deserve closer analysis.

This does not mean the output is a buy or sell signal. It is a screening tool. The final result should still be reviewed with price action, liquidity, fundamentals, dividend yield, dividend sustainability, and market conditions.

The value of this notebook is that it turns two separate API sources into a structured IDX dividend stock screener.

Conclusion

This tutorial shows how to build an IDX dividend stock screener using Python, getDividendCalendar, and getTrendingStocks.

The notebook retrieves dividend calendar data and trending stock data, converts them into dataframes, detects relevant columns, cleans both datasets, merges them by ticker, calculates normalized scores, and creates a final combined_score.

The final result helps identify stocks that are currently trending and also connected to dividend calendar data.

For traders, investors, and data enthusiasts, this workflow can become a useful starting point for building a dividend-aware stock screening system in the Indonesian market.

In the end, the project is not just about calling two APIs. It is about building a practical data-driven screener that combines market attention and dividend context into one ranked output.