Idea - AI Assisted Stock Market Sentiment Analysis using AFR Data and GPT-4

General Advise Warning The information on this site is not investment advice. It is general information only. It is not intended as financial advice or any other advice. It is not advice of any kind. We do not give advice. If you act on this information you must do your own research and seek advice from a independant advisor.

Introduction

Today I read "Can ChatGPT Forecast Stock Price Movements? Return Predictability and Large Language Models".

The study explores the capability of ChatGPT and other large language models for predicting stock market returns by conducting sentiment analysis of news headlines. ChatGPT was used to identify if a headline was positive, negative, or irrelevant for a company's stock price. Each evaluation was then assigned a numerical score. The results showed a positive correlation between these scores, termed "ChatGPT scores," and subsequent daily stock market returns. Interestingly, ChatGPT outperformed traditional sentiment analysis approaches.

The prompt they used was: Forget all your previous instructions. Pretend you are a financial expert. You are a financial expert with stock recommendation experience. Answer “YES” if good news, “NO” if bad news, or “UNKNOWN” if uncertain in the first line. Then elaborate with one short and concise sentence on the next line. Is this headline good or bad for the stock price of {company name} in the {term} term? Headline: {headline}

I thought, well ChatGPT can now connect to the web. So decided to try a very basic prompt using AFR Companies as a source of headlines as I am Australian and aware of the Australian market. I used the following prompt:

Visit URL: https://www.afr.com/companies get all of the headlines from the page for each article, for example "Universal Store shares tank 25pc on dour outlook", identify the company, for example "Universal Store", for each headline pretend you are a financial expert. You are a financial expert with stock recommendation experience. Answer “YES” if good news, “NO” if bad news, or “UNKNOWN” if uncertain in the first line. Then elaborate with one short and concise sentence on the next line. Is this headline good or bad for the stock price of the company name in the long-term

The results:

SkyCity

NO

The ongoing dispute with Macquarie over carpark operations, with no resolution in sight for the current financial year, suggests potential financial and operational challenges for SkyCity​​.

Universal Store

NO

A significant drop in shares, coupled with a pessimistic outlook, indicates unfavorable market conditions for Universal Store, which could negatively impact its long-term stock price​​.

Westpac

YES

Westpac's initiative to expedite mortgage approvals through the open banking regime could streamline operations, enhance customer experience, and potentially lead to increased revenues in the long term​​.

Webjet

UNKNOWN

Although Webjet's full-year earnings surpassed market estimates, the company's future performance will depend on its ability to navigate global economic challenges. This creates uncertainty regarding its long-term stock price​.

Redox

YES

The expansion of Redox's IPO broker syndicate suggests robust preparation for its anticipated billion-dollar IPO, which could potentially attract significant investor interest and positively impact its long-term stock price​​.

Critical Minerals Sector

UNKNOWN

The competition among rich nations for scarce resources like critical minerals could lead to price volatility and regulatory challenges, resulting in uncertainty for companies operating in this sector​​.

This was is great. it was able to fetch the headlines and process them, very slowly.

This is something that could be tested on a larger scale. So I am going to try and build a system that simulates a retail investor using GPT-4 to analyse the sentiment of AFR articles and predict the stock market in the long term in order to make investment decisions. I want it to playout like a real retail investor with little risk tollerence (no shorting, no borrowing), and it will only be able to use the information available to a retail investor.

Method:

1. Fetch the headlines from the AFR Companies page every 5 minutes.
2. For each of the headlines, use GPT-4 to analyse the sentiment of the headline and predict the stock market in the long term, identify the company and identify the ASX code.
3. Get the current price of the stock from the ASX, noting it is a 20 minute delay that needs to be factored in (i.e. the trade needs to occur at the delayed price 20 minutes of being aware of the article).
4. Start the investment simulation with $100,000.
5. If the sentiment is positive, buy the stock at the current price. If the sentiment is negative, sell the stock at the current price only if it is in your portfilio. If the sentiment is unknown, do nothing.
6. The buy trades have a ceiling of $1,000, can only buy whole units, and will have a transaction fee of $10.
7. The sell trades sell all the units, and will have a transaction fee of:
   • $10 if the units are less than $1,000
   • $19.95 (Over $1,000 up to $10,000 (inclusive))
   • $29.95 (Over $10,000 up to $25,000 (inclusive))
   • 0.12% (Over $25,000)
8. The trades can only occur during market hours (10am to 4pm), noting the 20 minute price delay and some ASX neuances.
9. Trades cannot occur on weekends or public holidays.
10. Trades cannot occur if the stock is suspended.

To keep it intesting I will run a leaderboard of different bots starting on different days. The leaderboard will be based on the total value of the portfolio at the end of each day. This gives the opportunity for a bot to recover from a bad start.

Potential Issues

• The fetching of data could fail or be blocked.
• Bias in the headings