I don't see virtualY: true in your init params.
There are 2 different phases before grid displays data: 1) Data loading & 2) Data binding.
Data loading is loading of data in browser from server ( usually via $.ajax or $.getJSON or grid's dataModel.location =="remote" ) and data binding is display of data in the grid when the required data is available in browser's memory.
When virtualX: true & virtualY: true is used, data binding shouldn't take roughly more than a second irrespective of the number of rows or columns.
Slow loading of data ( data loading ) over the network can be made fast by loading data in chunks, say if you have 5 - 7 thousand rows, try to load 500 - 1000 records at a time.
Other factors which can make it slow:
1) use of freeze* options in non virtual mode especially when the number of rows / columns is large.
2) use of inefficient or too much code in column.render callbacks.
3) use of inefficient or too much code in refresh event.
4) use of virtualXHeader: false can slow down the grid a bit if there are large number of columns.