The amplify-and-forward (AF) relay beamforming problems are naturally formulated as indefinite quadratic (non-convex) optimization programs. The typical methods for solving such optimization problems are to transform them into convex semi-definite programs (SDPs) with additional rank-one (non-convex and discontinuous) constraints. The rank-one constraints are then dropped to obtain solvable SDP relaxed problems and randomization techniques are employed for seeking the feasible solutions to the original nonconvex optimization problems. In many conventional scenarios, the results from solving the rank-one relaxed SDP problems are enough to conclude the solutions since no rank-higher-than-one SDP resulting matrix can be observed. Through our simulations, we found that there are also many scenarios that the SDP solvers give high-rank solutions. Hence, in this paper the rank-one constraints are equivalently expressed as reverse convex constraints and are incorporated into the optimization problems. Then, we propose an efficient iterative algorithm for solving the nonsmooth reverse convex optimization problems.Our simulations show that our proposed approach yields nearly global optimal solutions.