Caliga Hall was established before the civil war. Utilising slave labor, the Grays made a fortune off of tobacco and cotton production. Following the Civil War, however, the fortune slowly declined due to the abolition of slavery, and although the family still had wealth and power, they never reclaimed quite the same status.
The plantation itself consists of the main mansion house, several support buildings including a stable, a number of small houses for farmhands, and numerous acres of fields.
The plantation is strategically located near the town of Rhodes and several of the main transportation arteries to Saint Denis. This allows the Gray family to exert influence over the area, particularly in Rhodes, where they have a great deal of control over.
Caliga Hall is first visited around the beginning of the third chapter, after Dutch asks Arthur to investigate it and find out about about the Gray family. Arthur meets Beau Gray there, a young member of the family and the son of its patriarch, Tavish Gray. Beau asks Arthur to deliver a letter to his lover, Penelope Braithwaite - a member of the rival Braithwaite family.
Arthur later returns to Beau for his payment. While Beau pays him as promised, he reads Penelope's letter and becomes alarmed when he hears that she will be attending a Women's Suffrage rally in Rhodes, so he asks Arthur to ride out with him and protect her from any danger.
As part of the Van der Linde gang's plan to rob both families - the Grays and the Braithwaites - Arthur meets John Marston and Javier Escuella at Caliga Hall, where they are in conversation with Tavish Gray. The elderly Gray tells them that they can steal the Braithwaite family's prize horses and sell them to a nearby trader for $5000. The three proceed with this, but later discover that Tavish's assertion was wrong when they are paid only $700.
Around the same time, Hosea enters talks with the Braithwaites for jobs that they can do for them. The matriarch, Catherine Braithwaite, asks them to burn down the tobacco fields at Caliga Hall. To do this, Sean approaches Caliga Hall in a wagon stocked with moonshine and with Arthur in the back. A guard accompanies them to a warehouse, where either Arthur or Sean silently dispose of him and prepare for the arson. With a jug of moonshine, Arthur dowses the tobacco while Sean does the warehouses, and when everything is coated in moonshine, Arthur and Sean set fire to the plantation with fire bottles. The duo then proceed to escape Caliga Hall, which they successfully do after fighting off the incoming guards and finding payroll in an isolated wagon on the perimeter.
After the conclusion of the third chapter and the extermination of both the Gray family and the Braithwaite family, the player can revisit Caliga Hall. On the porch to the east of the main hall, the player can find the corpse of Tavish Gray, who committed suicide after receiving a letter from an Edinburgh University historian informing him that his ancestor was not an exiled Jacobite as previously believed, but a spy for the Duke of Cumberland.
- Near the northwest pier is a shack with a lean-to. On the workbench inside is the Cigarette Card for the Horse Crippler Cactus.
- On a workbench in the middle of the barn, centrally located in the area, is the Cigarette Card for the Manned Glider - 1893 - Charles Kinnear.
- To the south is a row of buildings. The second from the left, when looking at the map, has the Cigarette Card for New York on the front porch.
- Far to the southeast, just across the railroad tracks is an abandoned factory. On a windowsill on the upper platform is Valerian Root.
- Following the road southeast of the factory, just about at the curve, the Withered Arm can be found on the eastern side.
- Southeast of Caliga where the railroad crosses the river into Saint Denis, the Legendary Lake Sturgeon can be fished.
- Due south of the fishing location is a grouping of three islands. On the southeastern shore of the northern-most island is a wrecked boat with the Broken Pirate Sword.
- In the cornfields to the west, a scarecrow wearing the Scarecrow Sombrero can be found.
- The manor and surroundings are an almost perfect recreation of the real-world Shirley Plantation.