In medium saucepan, whisk together brown sugar, cocoa and cornstarch. Gradually mix in milk until well blended. Bring to a boil, stirring constantly, until thickened, about 1 minute. Remove from heat and whisk in vanilla.
Serve right away or pour into airtight container and refrigerate up to 1 week. Reheat in saucepan over low heat, stirring, or microwave in microwave-safe bowl, loosely covered, on medium-low power, 1 to 2 minutes.