Possible meanings are 1) only two or three prophets speak at any one meeting or 2) only two or three prophets take turns speaking at any one time.
"to what they say"