ChatGPT解决这个技术问题 Extra ChatGPT

How to concatenate two layers in keras?

I have an example of a neural network with two layers. The first layer takes two arguments and has one output. The second should take one argument as result of the first layer and one additional argument. It should looks like this:

x1  x2  x3
 \  /   /
  y1   /
   \  /
    y2

So, I'd created a model with two layers and tried to merge them but it returns an error: The first layer in a Sequential model must get an "input_shape" or "batch_input_shape" argument. on the line result.add(merged).

Model:

first = Sequential()
first.add(Dense(1, input_shape=(2,), activation='sigmoid'))

second = Sequential()
second.add(Dense(1, input_shape=(1,), activation='sigmoid'))

result = Sequential()
merged = Concatenate([first, second])
ada_grad = Adagrad(lr=0.1, epsilon=1e-08, decay=0.0)
result.add(merged)
result.compile(optimizer=ada_grad, loss=_loss_tensor, metrics=['accuracy'])
I think this problem is known as hierarchical fusion in AI, mostly used for multimodal data.

p
parsethis

You're getting the error because result defined as Sequential() is just a container for the model and you have not defined an input for it.

Given what you're trying to build set result to take the third input x3.

first = Sequential()
first.add(Dense(1, input_shape=(2,), activation='sigmoid'))

second = Sequential()
second.add(Dense(1, input_shape=(1,), activation='sigmoid'))

third = Sequential()
# of course you must provide the input to result which will be your x3
third.add(Dense(1, input_shape=(1,), activation='sigmoid'))

# lets say you add a few more layers to first and second.
# concatenate them
merged = Concatenate([first, second])

# then concatenate the two outputs

result = Concatenate([merged,  third])

ada_grad = Adagrad(lr=0.1, epsilon=1e-08, decay=0.0)

result.compile(optimizer=ada_grad, loss='binary_crossentropy',
               metrics=['accuracy'])

However, my preferred way of building a model that has this type of input structure would be to use the functional api.

Here is an implementation of your requirements to get you started:

from keras.models import Model
from keras.layers import Concatenate, Dense, LSTM, Input, concatenate
from keras.optimizers import Adagrad

first_input = Input(shape=(2, ))
first_dense = Dense(1, )(first_input)

second_input = Input(shape=(2, ))
second_dense = Dense(1, )(second_input)

merge_one = concatenate([first_dense, second_dense])

third_input = Input(shape=(1, ))
merge_two = concatenate([merge_one, third_input])

model = Model(inputs=[first_input, second_input, third_input], outputs=merge_two)
ada_grad = Adagrad(lr=0.1, epsilon=1e-08, decay=0.0)
model.compile(optimizer=ada_grad, loss='binary_crossentropy',
               metrics=['accuracy'])

To answer the question in the comments:

How are result and merged connected? Assuming you mean how are they concatenated.

Concatenation works like this:

  a        b         c
a b c   g h i    a b c g h i
d e f   j k l    d e f j k l

i.e rows are just joined.

Now, x1 is input to first, x2 is input into second and x3 input into third.


How are result and merged (or merged2) layers connected with each other on the first part of your answer?
and the second question. As I understand x1 and x2 is an input for first_input, x3 for third_input. What's about second_input?
second_input is passed through an Dense layer and is concatenated with first_input which also was passed through a Dense layer. third_input is passed through a dense layer and the concatenated with the result of the previous concatenation (merged)
@putonspectacles The second way using the functional API works, however, the first way using a Sequential-model is not working for me in Keras 2.0.2. I've roughly checked the implementation and calling "Concatenate([...])" does not do much and furthermore, you cannot add it to a sequential model. I actually think one still needs to use the depricated method "Merge([...], 'concat')" until they update Keras. What do you think?
What is the difference between Concatenate() and concatenate() layers in Keras?
P
Praveen Kulkarni

Adding to the above-accepted answer so that it helps those who are using tensorflow 2.0


import tensorflow as tf

# some data
c1 = tf.constant([[1, 1, 1], [2, 2, 2]], dtype=tf.float32)
c2 = tf.constant([[2, 2, 2], [3, 3, 3]], dtype=tf.float32)
c3 = tf.constant([[3, 3, 3], [4, 4, 4]], dtype=tf.float32)

# bake layers x1, x2, x3
x1 = tf.keras.layers.Dense(10)(c1)
x2 = tf.keras.layers.Dense(10)(c2)
x3 = tf.keras.layers.Dense(10)(c3)

# merged layer y1
y1 = tf.keras.layers.Concatenate(axis=1)([x1, x2])

# merged layer y2
y2 = tf.keras.layers.Concatenate(axis=1)([y1, x3])

# print info
print("-"*30)
print("x1", x1.shape, "x2", x2.shape, "x3", x3.shape)
print("y1", y1.shape)
print("y2", y2.shape)
print("-"*30)

Result:

------------------------------
x1 (2, 10) x2 (2, 10) x3 (2, 10)
y1 (2, 20)
y2 (2, 30)
------------------------------

o
o0omycomputero0o

You can experiment with model.summary() (notice the concatenate_XX (Concatenate) layer size)

# merge samples, two input must be same shape
inp1 = Input(shape=(10,32))
inp2 = Input(shape=(10,32))
cc1 = concatenate([inp1, inp2],axis=0) # Merge data must same row column
output = Dense(30, activation='relu')(cc1)
model = Model(inputs=[inp1, inp2], outputs=output)
model.summary()

# merge row must same column size
inp1 = Input(shape=(20,10))
inp2 = Input(shape=(32,10))
cc1 = concatenate([inp1, inp2],axis=1)
output = Dense(30, activation='relu')(cc1)
model = Model(inputs=[inp1, inp2], outputs=output)
model.summary()

# merge column must same row size
inp1 = Input(shape=(10,20))
inp2 = Input(shape=(10,32))
cc1 = concatenate([inp1, inp2],axis=1)
output = Dense(30, activation='relu')(cc1)
model = Model(inputs=[inp1, inp2], outputs=output)
model.summary()

You can view notebook here for detail: https://nbviewer.jupyter.org/github/anhhh11/DeepLearning/blob/master/Concanate_two_layer_keras.ipynb


What is the difference between Concatenate() and concatenate() layers in Keras?
Did you figure out the difference, one is a Keras class and another is a tensorflow method