前段时间尝试在XBMC的框架中添加对Airplay Screen Mirror的功能,有关Airplay的协议可以参考(当然是第三方破解的)
http://nto.github.com/AirPlay.html
本文指针对AAC-ELD音频的解析做一定说明,对于Airplay Screen Mirror本身暂不扩展。
如果是普通的AAC音频,自然可以使用FAAD的库进行解码,或者直接使用ffmpeg,网上有很多的资料,不过FAAD解不了AAC-ELD等级的AAC音频。
但问题有两点:
第一是流媒体的格式的音频,也就是没有文件封装格式,流媒体本身是通过RTSP来传递一些配置信息,再利用RTP来传输数据。
第二,AAC-ELD的编码格式,目前能参考的资料不多,不过最新版的ffmepg(>ffmpeg-1.0)或者libav,已经支持了AAC-ELD格式的编码,可以查看libavcodec目录下是否有libfdk-aacenc.c文件,其实本质是添加了对libfdk-aac音频库的支持,不过遗憾的是没有提供解码的代码。
其实,有关libfdk-aac解码的代码是有的,只是ffmepg没有将其吸纳进来,可以参考github上第三方修改的libav库,地址是https://github.com/Arcen/libav, 同时fdk-aac的库地址是https://github.com/Arcen/fdk-aac
可以看出libavcodec目录下已经有了libfdk-aacdec.c文件,网友可以自行下载编译,找一个音频文件,先将其转换成AAC-ELD格式的音频,然后在尝试解码。
编码的命令可以参考这种格式
ffmpeg -y -i test.wav -c libfdk_aac -profile:a aac_eld test.mp4
(注意:ffmpeg对于编码一般都采用第三方的库,例如x264等,但解码,ffmpeg本身会调用自己重写的解码代码,所以不一定就能够调用到libfdk-aac来解码AAC-ELD格式的音频,我当时是修改了ffmpeg的注册模块,强制利用fdk-aac来解码aac的音频来验证的)
以上解决的是文件封装的AAC-ELD音频,可以很容易验证,但对于RTP过来的裸音频数据,就要了解一下AAC的知识了,其中最最重要的是MPEG标准中规定,AAC格式的音频需要一个AudioSpecificConfig配置,如果你上面生成了mp4文件,就可以利用mp4info.exe这个工具查看esds字段(路径大概在trac->media->->minf->stbl->stsd->m4a->esds),然后可以参照esds的语法查找相应的AudioSpecificConfig字段。
下面贴一段参考代码,直接利用的fdk-aac的库,在ubuntu上运行的代码
(大概的流程是,有个线程在往队列里面填充数据,然后通过SDL播放,SDL的回调函数会去队列中取音频数据,然后调用fdk-aac解码,然后播放)
-
-
-
-
-
-
-
-
- #include <stdio.h>
- #include <stdlib.h>
- #include <SDL/SDL.h>
- #include <SDL/SDL_thread.h>
- #include <fdk-aac/aacdecoder_lib.h>
- #include "decodeAAC.h"
-
- #ifdef __MINGW32__
- #undef main /* Prevents SDL from overriding main() */
- #endif
-
- typedef unsigned char u8;
- typedef unsigned short int u16;
- typedef unsigned int u32;
-
-
-
-
-
-
- #ifdef ENABLE_PCM_SAVE
- FILE *pout = NULL;
- #endif
-
-
-
- static int fdk_flags = 0;
-
-
- #define N_SAMPLE 480
-
- UCHAR eld_conf[] = { 0xF8, 0xE8, 0x50, 0x00 };
- UCHAR *conf[] = { eld_conf };
- static UINT conf_len = sizeof(eld_conf);
-
- static HANDLE_AACDECODER phandle = NULL;
- static TRANSPORT_TYPE transportFmt = 0;
- static UINT nrOfLayers = 1;
- static CStreamInfo *aac_stream_info = NULL;
-
- static int pcm_pkt_size = 4 * N_SAMPLE;
-
-
-
-
- static int quit = 0;
-
- #define FDK_MAX_AUDIO_FRAME_SIZE 192000 //1 second of 48khz 32bit audio
- #define SDL_AUDIO_BUFFER_SIZE 4 * N_SAMPLE
- #define PCM_RATE 44100
- #define PCM_CHANNEL 2
-
- typedef struct AACPacket {
- unsigned char *data;
- unsigned int size;
- } AACPacket;
-
- typedef struct AACPacketList {
- AACPacket pkt;
- struct AACPacketList *next;
- } AACPacketList;
-
- typedef struct PacketQueue {
- AACPacketList *first_pkt, *last_pkt;
- int nb_packets;
- int size;
- SDL_mutex *mutex;
- SDL_cond *cond;
- } PacketQueue;
-
- static PacketQueue audioq;
-
-
-
-
- #define AAC_BUFFER_SIZE 1024 * 1024
- #define THRESHOLD 1 * 1024
-
- static u8 repo[AAC_BUFFER_SIZE] = {0};
- static u8 *repo_ptr = NULL;
-
-
-
- static void init_mem_repo(void)
- {
- repo_ptr = repo;
- }
-
-
-
-
- static void *alloc_pkt_buf(void)
- {
- int space;
-
- space = AAC_BUFFER_SIZE - (repo_ptr - repo);
-
- if (space < THRESHOLD) {
- repo_ptr = repo;
- return repo;
- }
-
- return repo_ptr;
- }
-
- static void set_pkt_size(int size)
- {
- repo_ptr += size;
- }
-
-
- static void packet_queue_init(PacketQueue *q)
- {
- memset(q, 0, sizeof(PacketQueue));
- q->mutex = SDL_CreateMutex();
- q->cond = SDL_CreateCond();
- }
-
- static int fdk_dup_packet(AACPacket *pkt)
- {
- u8 *repo_ptr;
-
- repo_ptr = alloc_pkt_buf();
- memcpy(repo_ptr, pkt->data, pkt->size);
- pkt->data = repo_ptr;
-
- set_pkt_size(pkt->size);
-
- return 0;
- }
-
- static int packet_queue_put(PacketQueue *q, AACPacket *pkt)
- {
-
- AACPacketList *pkt1;
-
-
- fdk_dup_packet(pkt);
-
- pkt1 = malloc(sizeof(AACPacketList));
- if (!pkt1)
- return -1;
- pkt1->pkt = *pkt;
- pkt1->next = NULL;
-
- SDL_LockMutex(q->mutex);
-
- if (!q->last_pkt)
- q->first_pkt = pkt1;
- else
- q->last_pkt->next = pkt1;
- q->last_pkt = pkt1;
- q->nb_packets++;
- q->size += pkt1->pkt.size;
-
- SDL_CondSignal(q->cond);
- SDL_UnlockMutex(q->mutex);
-
- return 0;
- }
-
-
-
-
- int decode_copy_aac_data(u8 *data, int size)
- {
- AACPacket pkt;
-
- pkt.data = data;
- pkt.size = size;
-
- packet_queue_put(&audioq, &pkt);
-
- return 0;
- }
-
- static int packet_queue_get(PacketQueue *q, AACPacket *pkt, int block)
- {
-
- AACPacketList *pkt1;
- int ret;
-
- SDL_LockMutex(q->mutex);
-
- for (;;) {
- if (quit) {
- ret = -1;
- break;
- }
-
- pkt1 = q->first_pkt;
- if (pkt1) {
- q->first_pkt = pkt1->next;
- if (!q->first_pkt)
- q->last_pkt = NULL;
- q->nb_packets--;
- q->size -= pkt1->pkt.size;
- *pkt = pkt1->pkt;
- free(pkt1);
- ret = 1;
- break;
- } else if (!block) {
- ret = 0;
- break;
- } else {
- SDL_CondWait(q->cond, q->mutex);
- }
- }
-
- SDL_UnlockMutex(q->mutex);
-
-
- return ret;
- }
-
-
-
-
- int fdk_decode_audio(INT_PCM *output_buf, int *output_size, u8 *buffer, int size)
- {
- int ret = 0;
- int pkt_size = size;
- UINT valid_size = size;
- UCHAR *input_buf[1] = {buffer};
-
-
- ret = aacDecoder_Fill(phandle, input_buf, &pkt_size, &valid_size);
- if (ret != AAC_DEC_OK) {
- fprintf(stderr, "Fill failed: %x\n", ret);
- *output_size = 0;
- return 0;
- }
-
-
- ret = aacDecoder_DecodeFrame(phandle, output_buf, pcm_pkt_size, fdk_flags);
- if (ret == AAC_DEC_NOT_ENOUGH_BITS) {
- fprintf(stderr, "not enough\n");
- *output_size = 0;
-
-
-
-
-
- }
- if (ret != AAC_DEC_OK) {
- fprintf(stderr, "aacDecoder_DecodeFrame : 0x%x\n", ret);
- *output_size = 0;
- return 0;
- }
-
- *output_size = pcm_pkt_size;
-
- #ifdef ENABLE_PCM_SAVE
- fwrite((u8 *)output_buf, 1, pcm_pkt_size, pout);
- #endif
-
- return (size - valid_size);
- }
-
- int audio_decode_frame(uint8_t *audio_buf, int buf_size)
- {
- static AACPacket pkt;
- static uint8_t *audio_pkt_data = NULL;
- static int audio_pkt_size = 0;
-
- int len1, data_size;
-
- for (;;) {
- while (audio_pkt_size > 0) {
- data_size = buf_size;
- len1 = fdk_decode_audio((INT_PCM *)audio_buf, &data_size,
- audio_pkt_data, audio_pkt_size);
- if (len1 < 0) {
-
- audio_pkt_size = 0;
- break;
- }
- audio_pkt_data += len1;
- audio_pkt_size -= len1;
- if (data_size <= 0) {
-
- continue;
- }
-
-
- return data_size;
- }
-
-
-
-
-
-
-
- if (quit) {
- return -1;
- }
-
- if (packet_queue_get(&audioq, &pkt, 1) < 0) {
- return -1;
- }
- audio_pkt_data = pkt.data;
- audio_pkt_size = pkt.size;
- }
- }
-
- void audio_callback(void *userdata, Uint8 *stream, int len)
- {
- int len1, audio_size;
-
- static uint8_t audio_buf[(FDK_MAX_AUDIO_FRAME_SIZE * 3) / 2];
- static unsigned int audio_buf_size = 0;
- static unsigned int audio_buf_index = 0;
-
-
-
- while (len > 0) {
- if (audio_buf_index >= audio_buf_size) {
-
-
- audio_size = audio_decode_frame(audio_buf, sizeof(audio_buf));
- if (audio_size < 0) {
-
- audio_buf_size = pcm_pkt_size;
- memset(audio_buf, 0, audio_buf_size);
- } else {
- audio_buf_size = audio_size;
- }
- audio_buf_index = 0;
- }
- len1 = audio_buf_size - audio_buf_index;
- if (len1 > len)
- len1 = len;
- memcpy(stream, (uint8_t *)audio_buf + audio_buf_index, len1);
- len -= len1;
- stream += len1;
- audio_buf_index += len1;
- }
- }
-
-
-
-
- void init_fdk_decoder(void)
- {
- int ret = 0;
-
- phandle = aacDecoder_Open(transportFmt, nrOfLayers);
- if (phandle == NULL) {
- printf("aacDecoder open faild!\n");
- exit(0);
- }
-
- printf("conf_len = %d\n", conf_len);
- ret = aacDecoder_ConfigRaw(phandle, conf, &conf_len);
- if (ret != AAC_DEC_OK) {
- fprintf(stderr, "Unable to set configRaw\n");
- exit(0);
- }
-
- aac_stream_info = aacDecoder_GetStreamInfo(phandle);
- if (aac_stream_info == NULL) {
- printf("aacDecoder_GetStreamInfo failed!\n");
- exit(0);
- }
- printf("> stream info: channel = %d\tsample_rate = %d
- \tframe_size = %d\taot = %d\tbitrate = %d\n", \
- aac_stream_info->channelConfig,
- aac_stream_info->aacSampleRate,
- aac_stream_info->aacSamplesPerFrame,
- aac_stream_info->aot,
- aac_stream_info->bitRate);
- }
-
-
-
-
- void init_fdk_aac_decode(void)
- {
- SDL_Event event;
- SDL_AudioSpec wanted_spec, spec;
-
-
- init_fdk_decoder();
- init_mem_repo();
-
-
-
-
-
-
-
- #ifdef ENABLE_PCM_SAVE
- pout = fopen("/home/juguofeng/work/star.pcm", "wb");
- if (pout == NULL) {
- fprintf(stderr, "open star.pcm file failed!\n");
- exit(1);
- }
- #endif
-
-
- wanted_spec.freq = PCM_RATE;
- wanted_spec.format = AUDIO_S16SYS;
- wanted_spec.channels = PCM_CHANNEL;
- wanted_spec.silence = 0;
- wanted_spec.samples = SDL_AUDIO_BUFFER_SIZE;
- wanted_spec.callback = audio_callback;
- wanted_spec.userdata = NULL;
-
- if (SDL_OpenAudio(&wanted_spec, &spec) < 0) {
- fprintf(stderr, "SDL_OpenAudio: %s\n", SDL_GetError());
-
- exit(1);
- }
-
- packet_queue_init(&audioq);
- SDL_PauseAudio(0);
-
-
-
- #if 0
- SDL_PollEvent(&event);
- switch(event.type) {
- case SDL_QUIT:
- quit = 1;
- SDL_Quit();
- exit(0);
- break;
- default:
- break;
- }
- #endif
-
-
- }
使用时,可以利用一下接口
void init_fdk_aac_decode(void);
int decode_copy_aac_data(unsigned char *inbuf, int size);
特别注意的是,RTP流中的AAC-ELD音频是裸数据,而解码器需要AudioSpecificConfig信息,这里我是自己事先知道了这个值。
由于离这个项目有段时间了,现在才将其罗列在这里,有些细节不是交代的很清楚,日后有空慢慢补充。
(当时由于对流媒体和mpeg4等标准不是很熟,走了很多的弯路,并且fdk-aac本身的教程只有文档说明,并没有一个代码实例,同时我实现的又是流媒体音频,所以有些坎坷,知道看到了第三方的libav中有了对AAC-ELD的解码支持的代码,在了解了AudioSpecificConfig的含义后,才成功解码了RTP中的AAC-ELD音频流)
(jgfntu) |