Is this a new set of hardware, or have you had focusing working fine with different software? It is important to confirm that the focuser is responding correctly to focuser movement commands from SGP. Your symptoms are consistent with the focuser not moving correctly in response to commands from the control program (SGP). Example: you move out 500 steps then in 500 steps, after which the physical focuser has not returned to its original position. I had a focuser once that did exactly this. There was a physical binding problem.
Here is how I would check this:
Open SGP and connect to your camera and focuser. Start with perfect focus using your mask. Take an image of 10 seconds using the Frame and Focus module. Click the Yellow star then make note of the "Half Flux Radius". You should see circles around many stars with the HFR for that star indicated. Make note of the HFR value.
In the Focus Control panel move the focuser OUT 200 steps. Watch the focuser when you click the OUT button to make sure the focuser actually moves out as you would expect it to. Take another image and check its HFR. It should be worse. If it is not at least 50% larger (worse) then try moving out another 500 steps and repeat the process. What we are attempting to find here is how many steps make the HFR 2 or 3 times larger than the best HFR at perfect focus. When you have found how many steps are required to move out of focus enough to double or triple the HFR value, Then move the focuser IN to the original position. Take another image and check its HFR. It should be very close to the original value. Also confirm visually that the focuser has returned the its original position. You could then repeat this entire process in the IN direction.
If this process gives you consistent and expected results, then next step is easy. Assuming to took 400 steps to double or triple the HFR, just divide this number by 4 giving 100. Use a Step Size of 100 and an Auto Focus Data Points of 9. The focusing routine will travel out 4 * Step Size = 400, and then move in 9 of these sets of 100 steps taking an image at each one. The plot of these 9 HFR values will ideally be an nice V curve.
In my case with my RC12 at 1960mm I use values of 80 and 9. But the best value for Step Size can vary widely depending on your focuser gear ratios and focuser motor stepping.