People are always asking to babysit or take my kids for the weekend and get upset when I say No?

We have two Children, an infant and a toddler. Both wonderful kids. Our family is constantly asking to take them for the weekend. Not just one member like 4 or 5 members. We go to all family functions and make it a point to get together when we are free. We see these family members about twice a month and we all get along great and I love them dearly. Problem is that they always ask to take the kids for the weekend. My husband and I have regular date nights and are very happy. We enjoy our weekends with our kids as we miss them all week while we work and they are in daycare. How do I get people to back off a bit. I allow a weekend once in awhile but this non stop badgering is driving me crazy and causing tension between our family. They are my family so my husband mostly stays out of it but now he is getting annoyed too :(