Error with JQuery

You have to understand how asynchronous calls work.

The validarCorreo function does not return any value. In fact it ends before the call to validar_correo.php is completed. The function makes the call and ends. Then, when the call is completed, the function indicated in success is executed. It is at that moment when you should check the received value and then, if appropriate, call the function insertar .

One possible implementation could be this:

function validarCorreo()
{
    var correo = $("#correoVal").val();

    $.ajax({
      type: "POST",
      url: "clases/validar_correo.php",
      data: "correo=" + correo, 
      success: function(data)
      {
        if(data > 0)
        {               
          $("#error_correo").show();
        }
        else
        {
         insertar();
        }
      }         
   });
}

function insertar()
{
  var nombre = $("#nombreVal").val();
  var correo = $("#correoVal").val();
  var password = $("#passwordVal").val();

  $.ajax({
   type: "POST",
   url: "clases/insertar_usuario.php",
   data: "nombre=" + nombre + "&correo=" + correo + "&password=" + password, 
   success: function(data)
   {
    if(data > 0)
    {             
     window.location.href = 'registro.php?id=' + data;
    }
    else
    {
      $("#error_cuenta").show();
    }
   }          
  });
}

Assuming that you receive a correct answer from your server and that the problem is numerical, the property / function success of $. ajax from jQuery is asynchronous , this means that you may or may not receive a data in the future, not instantly, therefore you need to use a way to collect your value / code to use when the data is ready. There are 3 extended ways to work asynchronously, with Callbacks (the one I'll show you) , with Promise and with Observable .

$(function() {

  validarCorreo(callback) {
    var nombre = $("#nombreVal").val();
    var correo = $("#correoVal").val();
    var password = $("#passwordVal").val();

    $.ajax({
      type: "POST",
      url: "clases/validar_correo.php",
      data: "correo=" + correo,
      success: function(data) {
        if (data > 0) {
          $("#error_correo").show();
          callback(false);
        } else {
          callback(true);
        }
      }
    });
  }

  validarCorreo(function(validated) {
    console.log(validated);
  });

});