Whisk pudding mix and milk for about 2 minutes. Pudding should start to thicken. Fold in the whipped topping or freshly whipped cream.
Arrange graham cracker squares across the bottom of an 8 inch baking dish. You may need to break some of them into small pieces to cover the bottom of the dish. Spread some of the pudding mixture across the graham crackers. Layer on more graham crackers and more pudding mix twice so that you have a total of 3 layers of graham crackers and two layers of pudding mixture.
To make the icing, beat the butter, cocoa powder and sugar together, then add milk and beat until creamy and smooth. Beat in vanilla.
Cover and chill for about 8 hours (I usually cut it sooner).