GPT-4 Fails Economics

Edit: Here is a new and improved version of the post below. I am leaving the original up for historical reasons, but I recommend following the link instead of reading this post.

*****************

Yesterday, I announced on this blog that ChatGPT had failed my economics exam. (All of the questions on this exam are taken from recent final exams in my sophomore-level course at the University of Rochester.) Multiple commenters suggested that perhaps the problem was that I was using an old version of ChatGPT.

I therefore attempted to upgrade to the state-of-the-art GPT-4, but upgrades are temporarily unavailable. Fortunately, our commenter John Faben, who has an existing subscription, offered to submit the exam for me.

The result: Whereas the older ChatGPT scored a flat zero (out of a possible 90), GPT-4 scored four points (out of the same possible 90). [I scored the 9 questions at 10 points each.] I think my students can stop worrying that their hard-won skills and knowledge will be outstripped by an AI program anytime soon.

(One minor note: On the actual exams, I tend to specify demand and supply curves by drawing pictures of them. I wasn’t sure how good the AI would be at reading those pictures, so I translated them into equations for the AI’s benefit. This seems to have had no deleterious effect. The AI had no problem reading the equations; all of its errors are due to fundamental misunderstandings of basic concepts.)

Herewith the exam questions, GPT-4’s answers (in typewriter font) and the scoring (in red):

Question 1: When the price of peanuts rises, Alice consumes fewer cashews. Can you determine whether cashews are a Giffen good for Alice?

To determine whether cashews are a Giffen good for Alice, we need to analyze her consumption behavior when the price of peanuts (a related good) rises. A Giffen good is a type of inferior good where the income effect outweighs the substitution effect, causing consumers to purchase more of the good when its price increases. In this case, when the price of peanuts rises, Alice consumes fewer cashews. This suggests that cashews are not a Giffen good for Alice, as her consumption of cashews decreases with the increase in the price of a related good.

Score: zero. The first sentence is wrong; to determine whether cashews are a Giffen good, we need to analyze her consumption behavior when the price of cashews (not peanuts) rises. From the given information, we need to draw an inference about what Alice would do if the price of cashews were to rise and make an argument to support that inference. Instead of an argument, all we are given here are the words “this suggests”. “This suggests” is not an argument.

Question 2: In a country where everyone is identical, 100 people wait in line each day to buy raspberries at a controlled price. The government has decided to hand out free coffee to the people standing in line. The coffee costs the government $1 per cup, but the people in line value that coffee at only 75 cents per cup. What is the social cost of providing the coffee?

In this scenario, the government hands out free coffee to the 100 people standing in line, with a cost of $1 per cup. However, the people in line value the coffee at only 75 cents per cup. To find the social cost, we need to compare the government's cost with the value people place on the coffee.

Government cost: 100 people * $1 per cup = $100
Value to people: 100 people * $0.75 per cup = $75

Social cost = Government cost - Value to people
Social cost = $100 - $75 = $25

The social cost of providing the coffee is $25.

Score: zero. This answer completely misses the key fact that free coffee will cause the line to get longer (in fact it must cause the line to get longer, given the stated assumption that everyone is identical, hence initially indifferent between standing in line and not standing in line). In fact (unless one assumes a very small population), the line must grow until the extra waiting time completely dissipates the value of the free coffee; thus the social cost of providing the coffee is $100.

Edited to add: See comments 14 and 15 below.

Question 3:
In the Fredonia housing market, supply is given by Q=P and demand is given by Q=100-4P, where P is the monthly rental rate of a one-family housing unit. The housing market is perfectly competitive.

In the grocery market, marginal cost is flat at $2 per pound. Each family’s monthly demand curve for groceries is given by Q=13-P.

The grocery market is currently perfectly competitive. If the grocery stores all merge and form a monopoly, how many families will leave town?

Score:
First, let's find the equilibrium price and quantity in the housing market.
Supply: Q = P
Demand: Q = 100 - 4P

Set supply equal to demand:
P = 100 - 4P
5P = 100
P = 20

Now, find the equilibrium quantity:
Q = 20 (from the supply equation)

In the housing market, there are 20 houses rented at a monthly rate of $20.

Now, let's analyze the grocery market. The demand curve for groceries per family is Q = 13 - P. If grocery stores merge and form a monopoly, they will maximize their profit by setting the price equal to the marginal cost, which is $2 per pound.

Q = 13 - 2 = 11 pounds

At a price of $2 per pound, each family will consume 11 pounds of groceries per month.

To determine how many families will leave town, we need to find the breaking point at which the cost of groceries becomes too high for families to stay. We don't have enough information to determine this directly, so we cannot accurately predict the number of families that will leave town due to the monopoly in the grocery market.

Score: zero. First, there are some extremely elementary errors, such as this howler: “If grocery stores merge and form a monopoly, they will maximize their profit by setting the price equal to the marginal cost, which is $2 per pound”. No, they will maximize their profit by raising the price from $2 per pound to $47.50 per pound, where marginal cost equals marginal revenue.

But I could forgive such errors if there were even an attempt to answer the question. The question is not even acknowledged until the final paragraph, where we are told that “we don’t have enough information” to find an answer. In other words, ChatGPT does not understand the basics of supply and demand. The increase in grocery prices reduces consumer surplus by $20 per family, which shifts the housing demand curve down by $20, leading to a new equilibrium quantity of 16 in the housing market; therefore 4 families leave town.

Edited to add: When I assigned this problem to students, it included one more bit of information: The new monopoly charges each customer a monopoly price (without regard to the effect on the town population). That information was inadvertently omitted from the question I submitted to the bot. Therefore a much better answer would be that the new monopolists foresee the effect of grocery prices on the town population, and therefore raise prices by a bit less than calculated above. I would nevertheless have given the bot full credit if it had failed to consider that effect.

Edited to add: See comments 17 and 18. There is an arithmetic mistake in my answer above, where the monopoly price should be $5.50 7.50, not $4. This is the sort of mistake for which I never penalize students. But the bot has now answered this problem twice, missing all of the economics each time. [For the record, this problem — like many of the problems — was presented to students with graphs instead of equations, and the numbers worked out quite a bit more comfortably. As always, I am penalizing the bot not for its inability to work with the particular equations but for its failure to grasp the essence of the problem.]

Question 4: You are the monopoly owner of a diner, where you can produce hamburgers at a marginal cost of $10 each. You can charge people for the privilege of entering the diner and buying hamburgers. The demand for hamburgers at your diner is given by Q=50-P, where P is the price in dollars and Q is the number of hamburgers per day.

Your customers don’t like to see each other eating, so each hamburger eaten imposes $5 worth of costs on other customers.

Suppose you can install curtains throughout the restaurant so customers don’t have to see each other. What is the maximum amount you’d be willing to pay for those curtains?

Without curtains:
Since you are a monopoly owner, you will set the marginal cost equal to the marginal revenue to maximize your profit. First, let's find the demand curve in terms of price:

P = 50 - Q

Total revenue (TR) is given by P * Q:

TR = (50 - Q) * Q = 50Q - Q^2

Now, let's find the marginal revenue (MR) by taking the derivative of the total revenue with respect to Q:

MR = d(TR)/dQ = 50 - 2Q

Now, set the marginal revenue equal to the marginal cost (MC = $10):

50 - 2Q = 10
2Q = 40
Q = 20

Now find the price:

P = 50 - 20 = 30

Total profit without curtains:

Profit = TR - TC = (P * Q) - (MC * Q) - (External Cost * Q)
External cost per hamburger is $5.

Profit = (30 * 20) - (10 * 20) - (5 * 20) = 600 - 200 - 100 = $300

With curtains:
Since there are no external costs with curtains, we can simply calculate the profit without considering the external costs.

Profit = TR - TC = (P * Q) - (MC * Q)
Profit = (30 * 20) - (10 * 20) = 600 - 200 = $400

The maximum amount you'd be willing to pay for curtains is the difference in profit with and without curtains:

$400 - $300 = $100

Score: zero. The first mistake is here: “Since you are a monopoly owner, you will set the marginal cost equal to the marginal revenue to maximize your profit”. No, you are a monopoly owner who can charge admission so you will set the marginal cost equal to the price. (Interestingly, this is exactly the opposite of one of the mistakes in the answer to Question 3. There, ChatGPT thought you’d set MC=P when you’d actually set MC=MR; here it thinks you’ll set MC=MR when you’d actually set MC=P.)

More egregiously, ChatGPT seems to have “accounted” for the externalities by subtracting them from the diner’s profit (!), failed to count admission fees as part of the profit, and most egregiously (because it misses the entire point) failed to even think about how the curtains are going to affect the demand for hamburgers.

Question 5: You are the monopoly owner of an unfenced amusement park where anyone can enter and purchase ride tickets. The demand for ride tickets is Q=100-5P and the marginal cost of providing the rides is P=4Q. You are thinking of building a fence so that you can charge admission to the park. What is the maximum amount you would pay for that fence?

First, let's find the optimal price and quantity for the amusement park without a fence. As a monopoly, you will set the marginal cost (MC) equal to the marginal revenue (MR) to maximize your profit.
Demand curve: Q = 100 - 5P
Inverse demand curve: P = (100 - Q) / 5

Total revenue (TR): TR = P * Q = Q * (100 - Q) / 5

Marginal revenue (MR): MR = d(TR)/dQ = (100 - 2Q) / 5

Marginal cost (MC): MC = 4Q

Now, set MR = MC:

(100 - 2Q) / 5 = 4Q
100 - 2Q = 20Q
100 = 22Q
Q = 100 / 22 ≈ 4.55

P = (100 - 4.55) / 5 ≈ 19.09

Total profit without a fence:

Profit = TR - TC = (P * Q) - (MC * Q)
Profit = (19.09 * 4.55) - (4 * 4.55^2) ≈ $43.44

Now, let's analyze the situation with a fence. Since you can charge admission, you can extract all the consumer surplus as additional profit. The consumer surplus is the area between the demand curve and the price line up to the quantity demanded.

Consumer surplus with a fence:

CS = 0.5 * (100 - 4.55) * 4.55 = 216.61

Total profit with a fence:

Profit = Profit_without_fence + CS
Profit = 43.44 + 216.61 = $260.05

The maximum amount you would pay for the fence is the difference in profit with and without the fence:

$260.05 - $43.44 = $216.61

The maximum amount you would be willing to pay for the fence is $216.61. This is because with the fence, you can charge admission and extract the consumer surplus, which increases your total profit.

Score: Two out of ten. First, mea culpa — I had intended to make the numbers easy to work with, but fortunately ChatGPT seems to have no problem with decimals. The calculation of profit without a fence is fine, but it is elementary material of the sort that students are supposed to know before they enter this course, so it earns no credit. ChatGPT realizes that with a fence you can extract the full consumer surplus, for which it gets two points out of ten. But it does not seem to have twigged to the main point, which is that when you can extract the consumer surplus, you have an incentive to increase that consumer surplus by lowering the price of ride tickets.

Edited to add: See comments 9 through 12 below.

Question 6: A monopoly-owned retail store is located next to a monopoly-owned parking lot. (They are owned by different monopolists, who do not collude.) Assume:

i) There is a fixed number of customers, all with identical downward-sloping demand curves for the products at the retail store.

ii) To shop at the store, you must park in the lot. There is no other reason to park in the lot.

iii) The store has an upward sloping marginal cost curve.

iv ) Parking spaces are provided at zero marginal cost.

a) Suppose the store announces a price, and then the parking lot announces a price. Describe how these prices are determined.

b) Suppose instead that the parking lot announces a price and then the store announces a price. Describe how these prices are determined.

c) Suppose instead that the prices are announced simultaneously. Describe how they are determined.

a) Store announces a price first, then the parking lot:

The store, being a monopoly, will set its price by equating its marginal revenue (MR) to its marginal cost (MC) based on the demand curve. This will maximize its profit.

Once the store's price is set, the parking lot owner knows the demand for parking spaces since parking is required for shopping at the store. Given that the parking spaces have zero marginal cost, the parking lot monopolist will set a price that captures all of the consumer surplus remaining after the store's pricing decision. This will maximize the parking lot's profit.

b) Parking lot announces a price first, then the store:

The parking lot owner knows that there's a fixed number of customers, and since parking spaces have zero marginal cost, the parking lot monopolist will set a price that extracts the maximum consumer surplus for parking.

The store owner, knowing the price of parking, will take that into account when setting its own price. The store will adjust its demand curve considering the price of parking (as it represents an additional cost for customers) and set its price by equating its MR to its MC based on the adjusted demand curve. This will maximize the store's profit.

c) Prices are announced simultaneously:

In this case, both the store and the parking lot monopolists will need to take each other's pricing decisions into account. They will try to set prices that maximize their individual profits.

The store owner will need to consider the potential impact of the parking price on demand and set its price by equating its MR to its MC based on an expected demand curve.

The parking lot owner will need to consider the potential impact of the store's price on parking demand and set its price to capture the expected consumer surplus remaining after the store's pricing decision.

In equilibrium, both the store and the parking lot will set their prices based on their expectations of each other's pricing decisions and the joint impact on consumer demand. The final prices will depend on the specific demand and cost functions of the store and the parking lot, as well as their expectations about each other's strategies.

Score: Score: two out of ten. The answer to a) is exactly correct. The answer to b) is wrong in multiple ways. First, customers who have paid for parking have exactly the same demand curves for store merchandise as customers who haven’t paid for parking, so the store does not “adjust its demand curve”. Second, the store will not “maximize its profit”, because doing so (given the price of parking) will drive all its customers away. Instead, the parking lot extracts the maximum possible consumer surplus, which occurs when the store’s producer surplus is zero. The store will then have to set prices so that producer surplus does equal zero in order to have any customers at all. The answer to c) looks like a lot of blather about what everyone will “consider” without enumerating the possible equilibrium outcomes. So full credit for a), no credit for b) or c). a) was the easy part, so two out of ten.

Question 7: True or false: If gasoline is demanded inelastically, then it’s more likely that the oil companies are colluding.

True: If gasoline is demanded inelastically, it means that changes in the price of gasoline have little impact on the quantity demanded. This situation is favorable for collusion among oil companies, as they can raise prices without significantly affecting the quantity demanded, resulting in higher profits. When demand is inelastic, consumers are less responsive to price changes, making it easier for colluding firms to maintain higher prices without losing significant market share.

Score: zero. This is exactly wrong. If gasoline is demanded inelastically, it cannot be priced monopolistically.

Question 8: Apples are provided by a monopolist. Pears are provided by a competitive industry. Coincidentally, the price of an apple is equal to the price of a pear. Suppose you want a piece of fruit, and you’d be equally happy with either an apple or a pear. If you care about social gain, which should you buy?

If you care about social gain, you should buy a pear. The reasoning behind this choice is that pears are provided by a competitive industry, where firms produce at the point where the price equals marginal cost, leading to allocative efficiency. In competitive markets, social surplus (the sum of consumer and producer surplus) is maximized.

On the other hand, apples are provided by a monopolist, who maximizes profit by producing at the point where marginal revenue equals marginal cost. This results in a higher price and a lower quantity produced compared to the competitive market outcome. Monopolists create deadweight loss, as they restrict output to maintain higher prices, leading to a reduction in social surplus.

By buying a pear, you support the competitive market, which promotes social gain and allocative efficiency.

Score: zero. This misses the point egregiously. The whole reason monopolies are inefficient is that they underproduce.
Competitive markets are already producing the right quantity, so buying a pear adds nothing to social gain at the margin.

Or say it this way: The marginal cost of producing an apple is less than its price; the marginal cost of producing a pear is equal to its price. Because the prices are the same, the apple must be cheaper to produce. Since they both make you equally happy, you’ll leave the world a richer place by buying the apple.

Question 9: You consume only X and Y . X is called a Pudgy good if your consumption of X rises whenever the price of Y rises.

a) True or False: Every inferior good is a Pudgy good.

b) True or False: Every Giffen good is a Pudgy good.

a) False: Every inferior good is not necessarily a Pudgy good. An inferior good is a type of good for which the demand decreases when the consumer's income increases. This definition is independent of the relationship between the price of another good and the consumption of the inferior good. A Pudgy good is one where the consumption of the good (X) increases when the price of another good (Y) increases. The relationship between the price of Y and the consumption of X is not a defining characteristic of inferior goods.

b) False: Every Giffen good is not necessarily a Pudgy good. A Giffen good is an inferior good with an unusual property: when its price increases, the quantity demanded also increases. This occurs because the income effect of a price increase for a Giffen good is so strong that it outweighs the substitution effect, leading to increased consumption. However, this definition does not imply that the consumption of a Giffen good will increase when the price of another good (Y) increases. The relationship between the price of Y and the consumption of X is not a defining characteristic of Giffen goods.

Score: zero. When the price of Y rises, the substitution effect leads you to consume more X. If X is inferior, the income effect works in the same direction, so your consumption of X is sure to rise, i.e. X is a pudgy good. For part b), if X is Giffen, then X is inferior, so part a) applies.