Take the target address family in account when determining the family of