You can start creative activities early with your kids and you really don't need lots of expensive materials.
Here are some ideas to stock your craft box and you can build on from there with bits and pieces as you plan different projects.
A good sized box that's easy to store is a great start and saves you having a clutter of paint and brushes stuffed into drawers. It also means you can pick it up and move it to different locations so if you're painting indoors or having fun outside you'll be able to pack up easily at the end.
Some tubs of paint in basic primary colours can be mixed to create other colours. Regular poster paint is the easiest to use and washes off little hands. Don't forget a paintbrush but you can just as easily paint with lots of other household items from potatoes to cotton wool or even just get busy with fingers.
Basic scrap paper is fine for early projects but you'll find that having some colour paper on hand will lead you into all sorts of creative projects.
If getting messy with paint scares you a little, then a set of basic crayons is mess free (unless your kids get near the walls).
Glueing and sticking things has always been a favourite with my kids. Some basic PVA while glue is suitable for most projects. I also collect interesting bits and pieces like sequins and buttons and store them in a box. I can then bring out the box of things to stick at a moment's notice.