A Nigerian woman has killed her husband, three children and herself in Makurdi, the capital of Benue state, the local police command has confirmed.