Is it permissible for us to sever ties of kinship if our blood-relatives` gatherings/meetings involve acts of sin?

Observing kinship ties is obligatory, and paying your blood-relatives a visit causing their sinful acts to be hindered , then you should do so. However, if their sinful acts continue while you are at their gathering and they didn't respond positively, then observing kinship ties via telephone and the like will suffice in this case. And Allah Knows 

When is Istikharah (guidance prayer) Du`a offered, before Tasleem (saying asalamu alikum warahmatu allah wa barakatuh to end the prayer), or after it?

It is offered after Tasleem.

Is it permissible for the big brother to force his sisters to wear Hijjab (Islamic wear) if their father didn`t do his part in this regard ?

All Perfect Praise is due to Allah, The Lord of The Worlds

The father`s attention should be drawn gently and politely, and the sisters should also be reminded with good words since Allah, The Almighty, Says (What means): "O ye who believe! Save yourselves and your families from a Fire whose fuel is Men and Stones." [Al-Tahreem/6]. And Allah Knows Best.

How is night prayer offered?

Night prayer is offering voluntary prayer after Maghrib and before Fajr (Dawn). As for Tahajjud, it is offering voluntary prayer at night after waking up voluntarily, and for Allah`s sake as He Says (What means): "And pray in the small watches of the morning: (it would be) an additional prayer (or spiritual profit) for thee: soon will thy Lord raise thee to a Station of Praise and Glory!" [Al-Isra`/79]. Offering Tahajjud is better than offering voluntary prayer before going to bed.