In a saucepan, combine first five ingredients, turn heat on medium, and cook, whisking constantly, for 10 minutes or thickened into a pudding consistency. Remove from heat and stir in vanilla extract. Let cool.
Thinly slice one banana and place the slices in the bottom of a deep dish, top with approximately 1/4 of the pudding, then add a layer of vanilla wafers. Repeat until you have four layers. Cover and chill for several hours (but preferably overnight) and serve.