Think of referential integrity as a data contract.
Let's say you create a relationship between Customer.CustID and Order.CustID.
Your Customer table makes an unbreakable promise that no row will be deleted if there are one or more related records in the Order table. Likewise, the Order table promises to only refer to records that actually exist in the Customer table.
With referential integrity in place, you can safely make assumptions about the relationships within your data. And that makes it easier to write your application.