# Application data timeline

This document follows data through the application in the order it is loaded,
refreshed, and changed. The application uses JSON files under `REPOSITORY/` as
its persistent store, JavaScript objects for the current browser session, and
MarketData.app as its market-data provider.

## Primary daily-open workflow (intended)

The daily archive refresh should run once at the start of the trading day, for
every ticker represented by a profile in `REPOSITORY/equityProfiles/`, not only
equities currently held in an account. It should:

1. Check each profile's earnings history and refresh it when the stored history
   is stale or does not cover the required period. Retain reported quarterly
   EPS and its report availability date/time in the equity profile. The initial
   history needs at least the two-year chart window plus four earlier reported
   quarters, so trailing-four-quarter EPS can be calculated at the start of the
   window.
2. Ensure the daily equity archive is current through the last completed
   trading session.
3. Use those closes and only earnings publicly available by each trading date
   to calculate the daily P/E series through the last completed session, and
   store it separately in `REPOSITORY/histPE/{TICKER}.json`. Do not use today's
   live quote as a substitute for a completed daily close.
4. Load the price history, moving averages, and historical P/E series for the
   dashboard chart.

This workflow is the intended primary refresh path; it is not yet implemented
as a single daily-open job. The existing earnings sync and price-history fetch
paths described below are narrower. The chart reads and displays an existing
historical P/E archive, but does not automatically build or refresh it.

## Timeline

### 1. Page load: accounts and holdings are read

`JS/domReady.js` initializes the app. `acctMngr.loadAcctData()` requests
`PHP/listAccounts.php`, which reads and merges the JSON files in
`REPOSITORY/ACCOUNTS/`.

Each account file can contain account information, equity holdings and their
transaction history, open options, and historical/closed options. The browser
keeps the merged accounts in `window.ACCT` and builds the in-memory `window.EQTY`
and `window.OPTN` objects from those holdings. These objects are session state;
the account JSON files are the persisted records.

The table is initialized from the account data, then the quote dispatcher runs.

### 2. Startup: current equity and option quotes are fetched

`dataMngr.dispatcher()` requests current quotes for the tickers in
`window.EQTY` from MarketData.app's bulk-quote endpoint. It requests each
account option's quote through `PHP/historicalDataManager_api.php`, then
updates the in-memory equity and option objects used to render the tables and
valuations.

This is a fetch when the page starts, not a background daily quote job. The
startup quote path does not write these quotes to the historical files.

### 3. Startup and dashboard: daily equity history is read or fetched

For each equity loaded from account holdings, startup initializes its
moving-average tracker by calling `equityMngr.getHistoricalData()`. Opening an
equity dashboard uses the same historical-data path.

The expected persistent file is `REPOSITORY/equityHist/{TICKER}.JSON`. If it
does not exist, the app asks the PHP API to fetch up to 400 daily candles
(`t`/timestamp and `c`/close) from MarketData.app. It calculates SMA20, SMA50,
SMA100, SMA200, and EMA21, and saves the timeline back to that file. A cached
timeline is normally reused. The code refetches when the cache cannot be read,
has fewer than 200 points, or lacks sufficient SMA200 values; the normal
existing-file path still has a TODO for checking whether its newest date needs
updating. This startup/on-demand path is not yet the daily-open archive refresh
for every ticker in `equityProfiles`.

Historical P/E has its own archive at `REPOSITORY/histPE/{TICKER}.json`; it is
not part of the price timeline. The series uses each trading day's close and
the trailing four reported EPS values available to investors by that date, and
covers up to the previous two years. The equity chart displays price and
moving-average lines in an upper panel and P/E in a lower panel, with aligned
dates and separate actual-value scales. Generate or refresh the archive from
stored prices and earnings with `php PHP/buildHistoricalPE.php TICKER`; the
script does not fetch or backfill source data. Automatic generation during the
morning refresh is not yet implemented.

Historical option files are stored separately under
`REPOSITORY/optionHist/`; option history is requested by chart/history features,
not by the startup quote dispatcher.

### 4. Earnings and profile data are synchronized

The current startup path synchronizes earnings for equities loaded from account
holdings. The API stores provider earnings data and sync metadata in that
equity's profile JSON under `REPOSITORY/equityProfiles/{TICKER}.json`. While the
page remains open, the earnings refresh loop repeats every six hours for those
same account equities. Adding a ticker also attempts an immediate sync.

The all-profile earnings backfill can be run with
`PHP/syncAllEquityEarnings.php`. It requests three years of earnings data,
merges returned quarters into each matching equity profile, and reports
whether it found at least eight reported quarters in the most recent two years
and four more before that window. Three years are requested because a two-year
daily P/E chart needs those earlier reported quarters to calculate trailing
EPS at its first date. The script is a manual backfill, not yet a scheduled
daily-open job.

Earnings must be applied according to their public report availability
date/time; a fiscal period end date alone is not sufficient to determine when
an EPS value can be used. P/E recalculation through the previous completed
session is not yet integrated into the daily refresh.

Equity profiles also hold descriptive/company information, dashboard and
indicator settings, and historical-data range/update metadata. Option profile
JSON files under `REPOSITORY/optionProfiles/` hold option details and
historical-range metadata where those features create/update them.

### 5. Add an equity: validate, register, create its profile, and open it

The Add Equity form validates the ticker with the provider, then posts it to
`PHP/addNewEquity.php`. The PHP endpoint:

1. Adds the ticker to `REPOSITORY/WATCHLISTS/equityWatchList.json`.
2. Creates `REPOSITORY/equityProfiles/{TICKER}.json` from
   `REPOSITORY/equityProfileTemplate.json`.
3. Attempts to synchronize earnings.

After success, the browser opens the equity dashboard, which loads/fetches its
daily history and syncs earnings. Adding a ticker does not by itself add it to
an account holding. Also, the startup dispatcher gets its live-quote symbols
from `window.EQTY` (built from account data), not directly from this watchlist.
The watchlist has a separate quote-fetch helper, but that helper is not the
startup dispatcher's input.

### 6. Add or edit an account holding or option

Account forms update the account data in `window.ACCT` and persist it through
`dataMngr.saveAccount()` / `PHP/saveAccount.php` to
`REPOSITORY/ACCOUNTS/{ACCOUNT_ID}.json`. Equity holdings and open option
positions therefore belong to the account record; quote and chart history
remain in their separate per-symbol files.

Options added to an account are represented by their OCC symbol under the
account's `openOptions`, grouped by underlying equity. Option profile/history
files are separate data products and are not the account's position record.
If an option's expiration date has passed, the user must acknowledge that it is
historical and provide its actual close date and signed close amount. It is
then recorded directly in `histOptions` with `value = book + closeAmount`.

### 7. Close an option

Closing an option updates the account position: a fully closed position is
removed from `openOptions`, and the closed details are recorded under
`histOptions`; a partial close updates the remaining open quantity and records
the closed portion. Each archived lot records `closeAmount` as the signed
allocated closing cash flow and `value` as the result (`book + closeAmount`),
alongside the closed quantity, opening book value, and dates. The changed
account is saved back to its account JSON file.

## Persistent data map

| Data | Persistent location | What it represents |
|---|---|---|
| Accounts and positions | `REPOSITORY/ACCOUNTS/*.json` | Account metadata, equity holdings/transactions, open and closed options |
| Equity watchlist | `REPOSITORY/WATCHLISTS/equityWatchList.json` | Tickers registered for watchlist use |
| Equity profile | `REPOSITORY/equityProfiles/*.json` | Company/profile fields, earnings, and update metadata |
| Equity daily history | `REPOSITORY/equityHist/*.JSON` | Daily timestamp/close timeline and moving averages |
| Option profile | `REPOSITORY/optionProfiles/*.json` | Option metadata and historical range |
| Option history | `REPOSITORY/optionHist/*.JSON` | Historical option price data |
| Current quotes and model objects | Browser memory (`window.ACCT`, `window.EQTY`, `window.OPTN`) | Session-only data used by tables, charts, and calculations |

## Refresh timing: implemented behavior vs. intended scheduler

The current page calls the quote dispatcher once during initialization. Earnings
sync repeats every six hours for account equities, and historical equity data
is fetched during startup or when an equity is opened, subject to the cache
rules above. The all-profile earnings backfill is currently a manual script. The intended
daily-open refresh should run it (or an equivalent scheduled operation), bring
daily closes current, then recalculate each equity's two-year P/E history
through the previous completed trading session before the dashboard chart
consumes prices, moving averages, and P/E together.

`JS/marketDataScheduler.js` defines a separate market-hours scheduler with a
three-minute default interval, but `index.html` does not load it and
`domReady.js` does not start it. Its real-time data methods are also placeholders
that return zero-valued prices. It should therefore not be treated as an active
source of regularly refreshed or persisted quotes.