For Paul had decided to sail past Ephesus

Paul sailed south past the port city of Ephesus, further south in order to land at Miletus.

so that he would not spend any time

"so that he would not have to remain for a time" or "so that he would not have a delay"