Improve MySql query coding

2

I have the following a doubt regarding the code that I implemented. Although it is functional, I am not convinced by the way in which I managed to solve the problem.

What it does is calculate the average of the recorded data at the exact times (ex: 08:00:00, 09:00:00) taking EL records of the previous 5 minutes, and EL of the 5 subsequent minutes. Ex: (05:55:00 + 06:00:00 + 06:05:00) / 3 and it gives me something like this

| temp |        fecha
|______|____________________
| 21.1 | 2018-12-06 06:00:00

Because I need the last 12 records with the previous calculation mentioned I do it by making 3 queries for the minutes specific and the last hours recorded ordering by date and time the query. After obtaining the 3 individual queries with limit of 12 and the first one with offset, in php process to take the records obtained, do the calculation and deliver an array similar to this:

| temp |        fecha
|______|____________________
| 21.1 | 2018-12-06 01:00:00
| 21.2 | 2018-12-06 02:00:00
| 21.3 | 2018-12-06 03:00:00
| 21.4 | 2018-12-06 04:00:00
| 21.5 | 2018-12-06 05:00:00
| 21.6 | 2018-12-06 06:00:00
| 21.7 | 2018-12-06 07:00:00

Although, this solution allows me to deliver the requested data, I would like to know a way to obtain everything in a single query, including calculating the average if possible.

This is the code that I have in the model.

(unfortunately, when the database was modeled, the date and time were separated: c)

as additional data after this goes to a controller that delivers the response as json to load it into a hightchart graph:)

$query1 = "SELECT temp,hora,fecha FROM estacion WHERE hora like '%:55:00' ORDER BY fecha DESC, hora DESC LIMIT 12 OFFSET 1"; 
$query2 = "SELECT temp,hora,fecha FROM estacion  WHERE hora like '%:00:00' ORDER BY fecha DESC, hora DESC LIMIT 12";
$query3 = "SELECT temp,hora,fecha FROM estacion  WHERE hora like '%:05:00' ORDER BY fecha DESC, hora DESC LIMIT 12"; 

$consulta1 = $this->db->query($query1);
$consulta1 = $consulta1->result_array();

$consulta2 = $this->db->query($query2);
$consulta2 = $consulta2 ->result_array() ;

$consulta3 = $this->db->query($query3);
$consulta3 = $consulta3 ->result_array() ;

foreach($consulta1 as $row => $index)
{
    date_default_timezone_set('UTC');
    //calcula temperatura
    $informacion = (($index[$dato])+($consulta2[$row][$dato])+($consulta3[$row][$dato]))/3;
    $informacion = number_format($informacion, 2, '.', '');
    //arregla fecha y hora
    $fecha = $consulta2[$row]['fecha'] . " " . $consulta2[$row]['hora'];
    $fecha = strtotime($fecha) * 1000;

    $datos[$row][0] = $fecha;
    $datos[$row][1] = $informacion;
}

//print_r( $datos );
//$consulta->result_array();
return $datos; 

I leave the create table in case it is necessary

CREATE TABLE 'estacion' ( 'fecha' date NOT NULL, 'hora' time NOT NULL, 'temp' float(3,1) DEFAULT NULL, 'humedad' int(3) DEFAULT NULL, 'pRocio' float(3,1) DEFAULT NULL, 'vPromedio' float(3,1) DEFAULT NULL, 'vRafaga' float(3,1) DEFAULT NULL, 'direcViento' int(3) DEFAULT NULL, 'precActual' float(5,1) DEFAULT NULL, 'precHoy' float(5,1) DEFAULT NULL, 'presion' float(7,2) DEFAULT NULL, 'precTotal' float(7,2) DEFAULT NULL, 'tempInterior' float(3,1) DEFAULT NULL, PRIMARY KEY ('fecha','hora') ) ENGINE=InnoDB DEFAULT CHARSET=latin1

example data

hora    | temp |   fecha
________|______|____________
03:55:00 | 21.1 | 2018-12-06
04:00:00 | 20.2 | 2018-12-06
04:05:00 | 24.2 | 2018-12-06
...
04:55:00 | 10.2 | 2018-12-06
05:00:00 | 12.2 | 2018-12-06
05:05:00 | 16.2 | 2018-12-06
...
05:55:00 | 22.2 | 2018-12-06
06:00:00 | 24.2 | 2018-12-06
06:05:00 | 26.2 | 2018-12-06
    
asked by Oscar Cid 06.12.2018 в 15:26
source

2 answers

1

After several days going around the issue, I managed to perform most of the operations in the query to the database, this including the transformation of the date with UNIX_TIMESTAMP .

Select
      ROUND(((DATO00.temp + DATO05.temp + DATO55.temp) / 3), 2) as Promedio
    , UNIX_TIMESTAMP
        (
            CONVERT_TZ(ADDTIME(CONVERT(DATO00.fecha, DATETIME), DATO00.hora),'+00:00','-03:00')
        ) * 1000 as FechaUNIX
from 
(
    SELECT temp,hora,fecha,ADDTIME(CONVERT(fecha, DATETIME), hora) as fechaCompleta
    FROM estacion
    WHERE hora like '%:00:00'
    ORDER BY fecha DESC, hora DESC 
) DATO00
INNER JOIN 
(
    SELECT temp,hora,fecha,ADDTIME(CONVERT(fecha, DATETIME), hora) as fechaCompleta
    FROM estacion
    WHERE hora like '%:05:00'
    ORDER BY fecha DESC, hora DESC
) DATO05
//se agregan 5 minutos al datetime
ON AddTime(DATO00.fechaCompleta, '00:05:00') = DATO05.fechaCompleta
INNER JOIN 
(
    SELECT temp,hora,fecha,ADDTIME(CONVERT(fecha, DATETIME), hora) as fechaCompleta
    FROM estacion
    WHERE hora like '%:55:00'
    ORDER BY fecha DESC, hora DESC
) DATO55
//se agregan 10 minutos al datetime
ON DATE_SUB(DATO05.fechaCompleta, INTERVAL 10 MINUTE) = DATO55.fechaCompleta
ORDER BY DATO00.fechaCompleta DESC LIMIT 12

Although, the logic of consulting in repeated occasions is maintained, they can be executed simultaneously by means of joins. In this way also, I could corroborate a serious error previous code, since the queries only asked for exact minutes in a differentiated way, and there was a case where one of the records was not found, generating inconsistency in the data and therefore, the average was miscalculated.

For example

hora     | temp |   hora   | temp |   hora   | temp |  Prom | 
_________|______|__________|______|__________|______|_______| 
00:55:00 | 12.1 | 01:00:00 | 15.1 | 01:05:00 | 17.1 | 14.77 | 
01:55:00 | 18.2 | 02:00:00 | 25.1 | 03:05:00 | 15.1 | 19.46 | 
02:55:00 | 17.2 | 03:00:00 | 22.1 | 04:05:00 | 13.1 | 17.47 | 

As can be seen, in the absence of the registration of 02:05:00 , proceeded to calculate the average with the following value, which corresponded to the registration of 03:05:00 as well as there were problems when calculating the average of 00:00:00 as the date in the database is not normalized.

The currently presented query avoids these problems, since when not finding those records, it does not list them thanks to the filters of the inner join that take the date now and subtract and add minutes depending on what I need.

Finally, I adapted the query using the class Query Builder where the 2nd method mentioned in the accepted response of this question was used to perform the join individually.

Then the query was parameterized with the information mentioned in this another question where it is mentioned that there is a way to prevent injections Escaping Queries

$estacion = 'estacion'; $dato = 'temp';

//Select de los %:00:00
$this->db   ->select($this->db->escape_str($dato).",hora,fecha,ADDTIME(CONVERT(fecha, DATETIME), hora) as fechaCompleta")
            ->from($this->db->escape_str($estacion))
            ->like('hora', ':00:00', 'before')
            ->order_by('fechaCompleta') ;

$Consulta00 = $this->db->_compile_select();
$this->db->_reset_select(); 

//Select de los %:05:00
$this->db   ->select($this->db->escape_str($dato).",hora,fecha,ADDTIME(CONVERT(fecha, DATETIME), hora) as fechaCompleta")
            ->from($this->db->escape_str($estacion))
            ->like('hora', ':05:00', 'before')
            ->order_by('fechaCompleta');

$Consulta05 = $this->db->_compile_select();
$this->db->_reset_select(); 

//Select de los %:55:00
$this->db   ->select($this->db->escape_str($dato).",hora,fecha,ADDTIME(CONVERT(fecha, DATETIME), hora) as fechaCompleta")
            ->from($this->db->escape_str($estacion))
            ->like('hora', ':55:00', 'before')
            ->order_by('fechaCompleta') ;

$Consulta55 = $this->db->_compile_select();
$this->db->_reset_select(); 

$query  = $this->db ->select("ROUND(((DATO00.".$this->db->escape_str($dato)." + DATO05.".$this->db->escape_str($dato)." + DATO55.".$this->db->escape_str($dato).") / 3), 2) as Promedio
                            , UNIX_TIMESTAMP
                                (
                                    CONVERT_TZ(DATO00.fechaCompleta,'+00:00','-03:00')
                                ) * 1000 as FechaUNIX"
                            )
                    ->from("($Consulta00) DATO00")
                    ->join("($Consulta05) DATO05","AddTime(DATO00.fechaCompleta, '00:05:00') = DATO05.fechaCompleta")
                    ->join("($Consulta55) DATO55","DATE_SUB(DATO05.fechaCompleta, INTERVAL 10 MINUTE) = DATO55.fechaCompleta")
                    ->order_by('DATO00.fechaCompleta DESC')
                    ->limit(12);

$query = $this->db->get();
    
answered by 12.12.2018 / 21:27
source
2

As commented by @acaedano, it is better to normalize your data, either in a temporary table (in memory) or in a separate table.

But another suggestion, somewhat cumbersome (by the formatting of dates and time) that you can use is this.

To save me the creation of a table in some database I will create a query on the flight of the form hour, temp, date with type query.

SELECT '%:36:00' as hora, '20.1' as temp, DATE('2018/12/06') as fecha
UNION
SELECT '%:37:00' as hora, '20.2' as temp, DATE('2018/12/06') as fecha
...

What will be like a subtable of the form:

hora    | temp |   fecha
________|______|____________
%:36:00 | 20.1 | 2018-12-06
%:37:00 | 20.2 | 2018-12-06

And I'll call it temp_table.

Then, from that subtable, I'll get a datetime object made from the concatenated and formatted date and time.

STR_TO_DATE(-- _______________________________________
    CONCAT(                                        -- |
        DATE_FORMAT(a.fecha, '%Y/%m/%d'),          -- |
        'T',                                       -- |_    (Arregla los formatos de fecha y hora)
        replace(a.hora,'%:','') -- Se elimina '%:' -- |     Se crea fecha para filtrar por rango.
    ), -- Junta hora y fecha.                      -- |     Se podría arreglar esto con una tabla
    '%Y/%m/%dT%i:%s'                               -- |   + con datos timestamp.
) as fecha_completa, -- ______________________________|

Creating datetime objects of the form '% Y /% m /% dT% i:% s', an example would be like this:

2018-12-06 00:36:00

As I did not understand what you wanted, I will divide the two points that I managed to understand of your request.

You want to do what your code does in PHP, literal.

That is, entering a time ( h ), as a center, and adding a range of radius 5, ie, [ h -5, h +5] you want to get AVG (center + ends)

Then this would be the query that you would store in a variable and to which you would later link the time to consult.

SELECT
    AVG(format_table.temp) as 'promedio'
FROM (
    SELECT
        STR_TO_DATE(
            CONCAT(
                DATE_FORMAT(temp_table.fecha, '%Y/%m/%d'),
                'T',
                replace(temp_table.hora, '%:', '')
            ),
            '%Y/%m/%dT%i:%s'
        ) as fecha_completa,
        temp
    FROM temp_table
) format_table
WHERE 
    -- Creas tu intervalo
    -- ___________________________________________________________________________________________________
    TIME(format_table.fecha_completa) = TIMESTAMP(
                                                DATE(fecha_completa), '00:45:00')
                    --                                                       |
                    --          Aqui 'bindeas' la fecha a consultar  ________|
    OR
    TIME(format_table.fecha_completa) = SUBDATE( -- La cota inferior
                                            TIMESTAMP(
                                                DATE(fecha_completa), '00:45:00'), INTERVAL 5 MINUTE)
                    --                                                       |
                    --          Aqui 'bindeas' la fecha a consultar  ________|
    OR
    TIME(format_table.fecha_completa) = ADDDATE( -- La cota superior
                                            TIMESTAMP(
                                                DATE(fecha_completa), '00:45:00'), INTERVAL 5 MINUTE)
                    --                                                       |
                    --          Aqui 'bindeas' la fecha a consultar  ________|
    -- ___________________________________________________________________________________________________
    -- Cierras tu intervalo

To try the query I did, something of this style runs.

CREATE TEMPORARY TABLE IF NOT EXISTS temp_table AS -- Tabla temporal por flojera.
    SELECT '%:35:00' as hora, '20.1' as temp, DATE('2018/12/06') as fecha
    UNION
    SELECT '%:37:00' as hora, '20.2' as temp, DATE('2018/12/06') as fecha
    UNION
    SELECT '%:38:00' as hora, '20.3' as temp, DATE('2018/12/06') as fecha
    UNION
    SELECT '%:39:00' as hora, '20.4' as temp, DATE('2018/12/06') as fecha
    UNION
    SELECT '%:40:00' as hora, '25.4' as temp, DATE('2018/12/06') as fecha
    UNION
    SELECT '%:41:00' as hora, '20.5' as temp, DATE('2018/12/06') as fecha
    UNION
    SELECT '%:42:00' as hora, '20.6' as temp, DATE('2018/12/06') as fecha
    UNION
    SELECT '%:43:00' as hora, '20.7' as temp, DATE('2018/12/06') as fecha
    UNION
    SELECT '%:44:00' as hora, '20.8' as temp, DATE('2018/12/06') as fecha
    UNION
    SELECT '%:45:00' as hora, '20.9' as temp, DATE('2018/12/06') as fecha
    UNION
    SELECT '%:46:00' as hora, '20.1' as temp, DATE('2018/12/06') as fecha
    UNION
    SELECT '%:47:00' as hora, '20.1' as temp, DATE('2018/12/06') as fecha
    UNION
    SELECT '%:49:00' as hora, '20.1' as temp, DATE('2018/12/06') as fecha
    UNION
    SELECT '%:50:00' as hora, '20.1' as temp, DATE('2018/12/06') as fecha
    UNION
    SELECT '%:52:00' as hora, '20.1' as temp, DATE('2018/12/06') as fecha;

SELECT
    AVG(format_table.temp) as 'promedio'
FROM (
    SELECT
        STR_TO_DATE(
            CONCAT(
                DATE_FORMAT(temp_table.fecha, '%Y/%m/%d'),
                'T',
                replace(temp_table.hora, '%:', '')
            ),
            '%Y/%m/%dT%i:%s'
        ) as fecha_completa,
        temp
    FROM temp_table
) format_table
WHERE
    TIME(format_table.fecha_completa) = TIMESTAMP(
                                                DATE(fecha_completa), '00:45:00')
    OR
    TIME(format_table.fecha_completa) = SUBDATE(
                                            TIMESTAMP(
                                                DATE(fecha_completa), '00:45:00'), INTERVAL 5 MINUTE)
    OR
    TIME(format_table.fecha_completa) = ADDDATE(
                                            TIMESTAMP(
                                                DATE(fecha_completa), '00:45:00'), INTERVAL 5 MINUTE)

Resulting in this style:

|      promedio      |
| 22.133333333333336 |

The other thing I understood was that you might want a new type table.

|      hora      |     promedio de los tres puntos importantes del intervalo    |

If so, this part has been killing me all day. I hope I can finish it soon.

    
answered by 07.12.2018 в 05:11