To better follow the energy consumption, the government wants energy suppliers to install smart meters in every home in England, Wales and Scotland. There are more than 26 million homes for the energy suppliers to get to, with the goal of every home having a smart meter by 2020.
This roll out of meter is lead by the European Union who asked all member governments to look at smart meters as part of measures to upgrade our energy supply and tackle climate change. After an initial study, the British government decided to adopt smart meters as part of their plan to update our ageing energy system.
In this dataset, you will find a refactorised version of the data from the London data store, that contains the energy consumption readings for a sample of 5,567 London Households that took part in the UK Power Networks led Low Carbon London project between November 2011 and February 2014. The data from the smart meters seems associated only to the electrical consumption.

目标： 综合考虑气候、时间、季节、节日以及需求响应等因素，实现负荷预测功能。

数据源： https://www.kaggle.com/jeanmidev/smart-meters-in-london

处理流程:

1. 将所有数据进行合并
2. 根据每户每天的能源消耗数据，对不一致的住户统计数据进行规范化处理
3. 探索天气状况等因素和能源消耗之间的关系
4. 将英国假日数据添加到日水平数据中作为指标
5. 拟合SARIMAX模型
6. 拟合LSTM模型

目录结构：

环境：

Keras==2.0.2
TensorFlow==1.15.5
scikit-learn==0.24.1

数据整合

def first():
    for num in range(0,112):
        df = pd.read_csv("data/daily_dataset/block_"+str(num)+".csv")
        df = df[['day','LCLid','energy_sum']]
        df.reset_index()
        df.to_csv("data/hc/hc_"+str(num)+".csv")

    fout= open("data/energy.csv","a")
    for line in open("data/hc/hc_0.csv"):
        fout.write(line)

    ###TODO 能源数据
    # 预测未来的能源需求，因此只取能源总量，即给定家庭每天的总能源使用量。
    for num in range(0,112):
        f = open("data/hc/hc_"+str(num)+".csv")
        f.readline() # skip the header
        for line in f:
             fout.write(line)
        f.close()
    fout.close()
first()

energy = pd.read_csv('data/energy.csv')
housecount = energy.groupby('day')[['LCLid']].nunique()
#print(housecount.head(4))
energy = energy.groupby('day')[['energy_sum']].sum()
energy = energy.merge(housecount, on = ['day'])
energy = energy.reset_index()
energy.count()
energy.day = pd.to_datetime(energy.day,format='%Y-%m-%d').dt.date
energy['avg_energy'] = energy['energy_sum']/energy['LCLid']
#print(energy.describe())

weather = pd.read_csv('data/weather_daily_darksky.csv')
print(weather.head(4))

# 每日级别的天气信息是使用数据集中的 darksky api
weather['day']=  pd.to_datetime(weather['time']) # day is given as timestamp
weather['day']=  pd.to_datetime(weather['day'],format='%Y%m%d').dt.date
# 选择数字变量
weather = weather[['temperatureMax', 'windBearing', 'dewPoint', 'cloudCover', 'windSpeed',
       'pressure', 'apparentTemperatureHigh', 'visibility', 'humidity',
       'apparentTemperatureLow', 'apparentTemperatureMax', 'uvIndex',
       'temperatureLow', 'temperatureMin', 'temperatureHigh',
       'apparentTemperatureMin', 'moonPhase','day']]
weather = weather.dropna()
### 天气状况与用电量的关系
weather_energy = energy.merge(weather,on='day')
#print(weather_energy.head(2))

#***1.  温度 ***
# 我们可以看到能量和温度有一个反比关系
# 在低温时，很可能通过加热器等增加能源消耗。
plt.rcParams['font.sans-serif'] = ['SimHei'] # 步骤一（替换sans-serif字体）
plt.rcParams['axes.unicode_minus'] = False   # 步骤二（解决坐标轴负数的负号显示问题）

def drow1():
    fig, ax1 = plt.subplots(figsize = (20,5))
    ax1.plot(weather_energy.day, weather_energy.temperatureMax, color = 'tab:orange')
    ax1.plot(weather_energy.day, weather_energy.temperatureMin, color = 'tab:pink')
    ax1.set_ylabel('Temperature')
    ax1.legend()
    ax2 = ax1.twinx()
    ax2.plot(weather_energy.day,weather_energy.avg_energy,color = 'tab:blue')
    ax2.set_ylabel('Average Energy/Household',color = 'tab:blue')
    ax2.legend(bbox_to_anchor=(0.0, 1.02, 1.0, 0.102))
    plt.title('能耗和温度')
    fig.tight_layout()
    plt.show()

#***2.  湿度 ***
# 湿度和平均能耗似乎有相同的趋势。
def drow2():
    fig, ax1 = plt.subplots(figsize = (20,5))
    ax1.plot(weather_energy.day, weather_energy.humidity, color = 'tab:orange')
    ax1.set_ylabel('Humidity',color = 'tab:orange')
    ax2 = ax1.twinx()
    ax2.plot(weather_energy.day,weather_energy.avg_energy,color = 'tab:blue')
    ax2.set_ylabel('Average Energy/Household',color = 'tab:blue')
    plt.title('能耗和湿度')
    fig.tight_layout()
    plt.show()

#***3.云层值  Cloud Cover***
def drow3():
    fig, ax1 = plt.subplots(figsize = (20,5))
    ax1.plot(weather_energy.day, weather_energy.cloudCover, color = 'tab:orange')
    ax1.set_ylabel('Cloud Cover',color = 'tab:orange')
    ax2 = ax1.twinx()
    ax2.plot(weather_energy.day,weather_energy.avg_energy,color = 'tab:blue')
    ax2.set_ylabel('Average Energy/Household',color = 'tab:blue')
    plt.title('Energy Consumption and Cloud Cover')
    fig.tight_layout()
    plt.show()

#***4.能见度 Visibility***
#> 能见度因素似乎不会影响能源消耗,
# 因为能见度很可能是一个户外因素，它的增加或减少不太可能影响家庭内部的能源消耗。
def drow4():
    fig, ax1 = plt.subplots(figsize = (20,5))
    ax1.plot(weather_energy.day, weather_energy.visibility, color = 'tab:orange')
    ax1.set_ylabel('Visibility',color = 'tab:orange')
    ax2 = ax1.twinx()
    ax2.plot(weather_energy.day,weather_energy.avg_energy,color = 'tab:blue')
    ax2.set_ylabel('Average Energy/Household',color = 'tab:blue')
    plt.title('Energy Consumption and Visibility')
    fig.tight_layout()
    plt.show()

#***5. 风速 Wind Speed***
#>  像能见度一样，风速似乎是一个户外因素，并不影响能源消耗。
def drow5():
    fig, ax1 = plt.subplots(figsize = (20,5))
    ax1.plot(weather_energy