King Charles and Tom Daley caught up over their shared love of knitting at a garden party in Washington